7zip and Gzip make bootable restores.

zip backups <span style="font-size: 16px;">can't restore to an SD card.  At the start of the restore process, the SD card is unmounted and then quickly remounted, and AP-B hangs.</span>

If the same .zip backup is expanded using another utility, then the image is restored to an SD card; it now works.  (I had problems with that in the past so one of the recent changes to AP-B fixed that.)

Irv

Thanks Irv!

That's an excellent find! 
I'll see if I can fix that,...!!!

OK, I'm making progress ...

The issue is two fold ...

1) 7za (7Zip) does not like a filename called "-" without some help
2) 7za cannot create a ZIP file from stdout (not supported for zip, works fine for GZip, 7Zip, etc)
3) Regular Zip can only create it as a file with the name "-" when making a backup
4) for some reason unzip is not cooperating either

At least I know where the problem is, so I'll be working on a fix for that.

I have just posted a new version of 1.9.1.
This should address the issue with restoring an image from a ZIP backup.

The issue was indeed with the weird filename used by ZIP, where it names the compressed file/image just "-", so I managed to find a way around that.
I suppose I should have tested that a little better.

This also includes a new version of 7za, which hopefully works. 
(not related to the solution as I had hoped, but the old one was just version 9 and this is version 15, so ... quite a fee version jumps forward)

You can download the test version here. 

The new version works with ZIP files for me.

Thanks,

Irv

Success on my second attempt at reducing the size of my master image so it will fit on all 32GB cards.  I reduced it to 29GB.

I used the instructions at:   http://softwarebakery.com/shrinking-images-on-linux  with a few issues.

1. These instructions were not written for a NOOBS version.  The first time, I only reduced the size of the root partition p7.  The resulting SD card seemed to work but running gparted gave me the  error: "Can't have a partition outside the disk".  I started over and reduced both the root partition p7 and the extended partition p2.  The SD card from that image doesn't show errors in gparted.  Warning:  I have a very limited understanding of NOOBS partitions so there may be undiscovered side effects.  I completed this an hour ago--so all I know is it boots and the Web browser works.

2. My master .img file was too large to fit onto any of the four 32GB cards that I have.  (The NOOBS card that came with my Rpi had more capacity than most.)  So I bought a 64GB card and had to format it as ExFat so it would support a 32GB file. (The same would be true for a 32GB card.)  Then I had to install ExFat support on my Rpi with: exfat-fuse. 

There may be an easier way to reduce the size of an image and SD card using the process discussed at: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=142987 .  It converts a NOOBS SD to a "Raspbian only" SD by getting rid of the extra NOOBS partitions which should reduce the size accordingly.  At first glance, this process could probably be done on the Mac and automated.  It seems to only use DD, formatting, and a change to config/fstab.

Irv

Thanks Irv,

double whammy on the good info! 

Good to know that ZIP now works correctly as well - thank you very very much for testing!
I guess I can release that version today or tomorrow then. I just arrived in Houston, and had packed the 2 32Gb SD cards, just in case .

As for shrinking partitions; those are great tips.
Wish I could do all this straight from the Mac though. 
I even tried contacting Paragon, since they do have some great apps to reduce partition size. Haven't heard from them so I guess they are not interested.

I will do some experiments when I get home again ... (in a few weeks)
Unfortunately, I do need to work to earn an income ... wish I could do this full time though hahah 

I've started an attempt at the second way, converting NOOBS to regular Raspbian on the Mac.  I'm having a problem with the destination path and all the examples I can find just write to the current directory.  I'd like to do this (but it gets rejected).  
sudo dd if=/dev/rdisk6s7 of=/dev/disk5/BUofRoot.dmg
---------
This works if my working directory is disk5:   sudo dd if=/dev/rdisk6s7 of=BUofRoot.dmg

Thanks Irv!

I can imagine you've been here only one day ... it has not impressed me so far, nice people though.

As for your resize issue ... writing the output file (of) to "/dev/disk5/BUofRoot.dmg" seems a little unusual, and that's probably the reason why "0f=BUofRoot.dmg" is working. I would normally write an output file to a mounted volume, instead of straight to a volume.

I'd assume those instructions are intended for use on Linux, and not MacOS X. 

Booting with GPartEd and modify the SD card with that is something that might work - however, moving from NOOBS to non-NOOBS might require a few extra steps. This is also the main reason why I stopped using NOOBS. Awesome for your first few runs with the Raspberry Pi, but pretty quick a little bit of a nightmare ... 

The process of converting a NOOBS SD card to a Raspbian-only SD card works as described at:   https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=142987     It  gets rid of the extra NOOBS partitions which reduces the file system size accordingly making it easier to backup and then restore to a slightly smaller SD card.  My first attempt works but needs a little fine tuning. Here's an outline:

Backup the boot and root partitions from the SD card to your Mac

• Use DD to copy /dev/mmcblkp06 (boot) from your SD card to boot.dmg on your Mac hard drive

<span style="font-size: 16px;">• Use DD to copy  /dev/mmcblkp07 (root) from your SD card to root.dmg on your Mac hard drive</span>

<b style="font-size: 16px;">

<b style="font-size: 16px;"> Format a "New" SD card with three partitions

  - boot with a size to match the boot partition on your SD card

  - root with a size to match the root partition on your SD card

  - the extra space as free space (Mine was about 2GB which should give room for it to fit on slightly small 32GB cards--but I haven't tested that yet.)

Restore the boot and root partitions from your Mac to the "New" SD card

• Use DD to copy boot.dmg from your Mac to the first partition on the SD card

• Use DD to copy root.dmg from your Mac to the second partition on the SD card

Boot your Rpi using an existing card and mount the "New" SD card in one of the USB slots

• Edit /boot/cmdline.txt on the "New" card changing:  partition p6 to p1 

• Edit /root/etc/fstab "New" card <span style="font-size: 16px;">changing: partition p7 to p2</span>

I did these edits on the Rpi because I couldn't mount root on my Mac.  I tried installing Ext2FS_1.4d4 but the install failed.

Shutdown and boot from the "New" card

Test it and check out its partitions with gparted

It would be a good idea to boot from another card and mount the "New" card in an USB slot and run e2fsck on the "New" card.

Fine tuning: 

• I made my root partition a little larger on the "New" card than needed. Gparted complains about that. This may not be important but there's no reason to waste the space.

• Gparted info on the boot partition says it's unreadable.

A lot more testing needs done but this approach looks promising in reducing the size of the filesystem on a 32GB card so it can be backed up and then restored to slightly smaller 32GB cards. 

If any part of this outline doesn't make sense, use the info in the link at the top of this post--that's what I used.

Irv

This is a follow up post to the ones immediately above re: converting an SD card from a seven partition NOOBS to a two partition plain Raspbian with unallocated free space at the end. This is a way around the problem of Apple Pi-Baker and DD complaining that the source image is larger than the destination SD card.

Read the outline in the post above before going farther. That gives you an overview of the process without a lot of confusing detail. This post will fill in the details.

Warning: I'm a Raspberry Pi and Linux newby. There may be better or less risky ways to accomplish these tasks. Consider this a proof of concept and not a polished methodology.

COPY BOOT & ROOT PARTITION IMAGES INDIVIDUALLY TO HARDDRIVE FROM SD CARD (ON MAC)

diskutil list

sudo diskutil unmount /dev/disk6s6 ## change to your SD card disk & partition numbers

cd /volumes/"Internal Raid" ## Change to your destination directory.

sudo dd if=/dev/rdisk6s6 of=boot.dmg bs=4096; sync

sudo dd if=/dev/rdisk6s7 of=root.dmg bs=8192; sync

CALCULATE BYTES NEEDED FOR THE NEW PARTIONS (ON Rpi)

<span style="font-size: 16px;">Put the card you just made images from back in your Rpi and use fdisk or gparted to look at the partition sizes.</span>

sudo fdisk -l /dev/sda

 Sectors used 56609800 * 512 bytes = 28984217600 bytes (This is my root)

If you have to calculate sectors used from the starting and ending sector numbers, don't forget to add 1 after subtracting.

boot 2488319 - 2359296 = 129023+ 1= 129024 * 512 = 66060288 bytes

FORMAT AND PARTITION SD CARD (ON MAC)

<span style="font-size: 16px;">Format SD card with 3 partions: two FAT and 1 Free Space.</span>

Disk utility is confusing when specifying the size of a partition. It uses a strange definition of GB. Set a partition to 29GB but it ends up as: 29,000,003,584 bytes when it should be 3113851289.

It's easiest to use the command line using bytes from your sector calculations and your SD card drive number from Disk Utility

diskutil partitionDisk /dev/disk6 MBRFormat MS-DOS BOOT 66060288B MS-DOS ROOT 29000003584B free FREE 0B

COPY INDIVIDUAL PARTION IMAGES FROM HARD DRIVE TO SD CARD on MAC

BOOT

cd /volumes/"Internal Raid" (Change to where you stored your images.)

diskutil list

sudo diskutil unmount /dev/disk6s1 (Change to your destination SD card as reported by diskutil.)

sudo dd if=boot.dmg of=/dev/disk6s1 bs=4096; sync (Change disk number)

ROOT

sudo diskutil unmount /dev/disk6s2 (Change disk number)

sudo dd if=root.dmg of=/dev/disk6s2 bs=4096; sync (Change disk number)

<span style="font-size: 16px;">This can take awhile. A 29GB root partition took me anywhere from 3 to 8 hours depending on the speed of the destination card.</span>

CHECK THE FILE SYTEM ON THE NEW CARD (ON Rpi)

Mount the new card in a USB slot on the Rpi.

umount /dev/sda1 (Check disk number matches your system)

sudo fsck -n /dev/sda1 (Check disk number)

No errors should be reported

umount /dev/sda2 (Check disk number)

sudo fsck -n /dev/sda2 (Check disk number)

No errors should be reported

Use gparted to look at the partitions on the new SD card.

sudo gparted

It should show unallocated space at the end of two partitions.

A boot partition of 63Mib (At least for my NOOBS distribution.)

A root partition whose size will vary. Mine shows 27.01GiB, 21.67 unused with 4.7GiB used. Mine has a warning that there is 656.56Mib of unallocated space within the partition. I don't know why that's there or the possibility it might cause a problem later. (Maybe if I filled up the 21.67 unused space.)

EDIT cmdline.text and fstab ON THE NEW CARD (ON Rpi).

With the new card mounted in a USB slot on the Rpi.

Edit /boot/cmdline.txt on the “New” card changing: partition p6 to p1 & partition p7 to p2

Edit /root/etc/fstab “New” card changing: partition p7 to p2

SHUTDOWN THE Rpi AND BOOT FROM YOUR NEW CARD (ON Rpi)

If it doesn't boot, look at the troubleshooting steps below.

If they're no help, make a post here with the last text showing on the display where it hangs up.

MAKE COPIES OF THE NEW SD CARD USING APPLE PI-BAKER (ON MAC)

Now you're ready to make a backup copy of the entire new card onto your harddrive using Apple Pi-Baker. After that, you can easily make (restore) as many clones as you want.  

Apple Pi-Baker will report during the restore that the image is too large for the card. That's OK, the bytes at the end of the copy that don't fit and are lost are part of the 1GB or more of unallocated space of the third partition.

TROUBLESHOOTING

If format fails with "Unable to write to the last block of the device." or "File System Formatter Failed", format TWICE in a camera. If that doesn't work try HP USB Disk Storage Format Tool - v2.1.8

< http://files.extremeoverclocking.com/file.php?f=197> The errors came up in The Disk Utility application. Formatting from the command line might have different error messages.

<span style="font-size: 16px;">----</span>

If dd finishes with "end of device". This seems to mean the copy stopped because it ran into the end of the partition. You probably made the partition to small--recheck your calculations.

---------

If you get "FAIlED Unknown block(179,1)" on Rpi bootup with the new card, it most likely means that Rpi can't find the root file system.

- check cmdline.txt and fstab to make sure you changed them correctly

- use gparted and make sure the root partition isn't pure white but shows an appropriate amount of beige content. I've see the pure white when I apparently didn't make the root partition large enough and the DD copy to the SD card ended with: "end of device":

Summary

There are two reasons to go through all this.

1. To convert a NOOBS SD card to plain Raspbian for whatever good reason you have.

2. To reduce the size of your NOOBS SD card so you can copy it to a slightly smaller SD card.  NOOBS has extra partitions.  Only two are needed.  The rest use roughly  1GB of space--most of it by the Recovery partition.

This is proof of concept that almost all the work can be done on the Mac and probably automated.  Editing two text files are the critical parts that are easier on the Rpi.

Have fun,

Irv Kanode

Hans,

In an earlier post, you asked if the "Restore" function should automatically repartition the card.  I think it should.  

There's no harm in it--the card is going to be overwritten anyhow.  It could even be somewhat safer in that the NOOBS repartition button would be seldom used.  Making it less likely that I will repartition my source card before using the Backup button.

Irv

Awesome info! 

As for the repartitioning,... the reason why I took it out, is because of the "start" delay when writing an IMG, and... technically repartitioning should not have an effect at all, since DD simply overwrites everything. So much for theory of course, because we all know that in practical situations things can work very different. I'll try to compile a version where things get repartitioned before writing an IMG and see what the response is of users.

Hi I installed NOOBS, installed a script to sense a shutdown button and finally installed a video looper. The video looper should, and does load at start up.

I then created a backup of this using PI BAKER on my mac asnd restored this image to a new identical micro sd card. I now find with the new card that it all boots fine, but doesn't autorun the adafruit video looper software it just goes to the pi desktop. Anybody know why?

Hi Kltmacd,

I cannot even begin to guess why this happens ... 
ApplePi-Baker does not change anything on the image, it makes a byte-by-byte copy (using "dd").
So the only two things I can come up with are:

- The other SD card is just a few critical bytes smaller ... (doubt it)
- The script somehow looks for something that's not there anymore?

Did you try another SD card?