Tests by Irv:

I gave up for awhile on trying to backup 32GB cards. I was successful in backing my master image to a 64GB card so it’s now less important.

However, I ran into the “hung” backup with some 8 and 16 GB cards. 

• I made a new NOOBS card and set Raspbian up the way I want it–ready to add applications

• I made a zip img backup.

• When I tried to clone it to other 8 or 16GB cards, it hung up at the start of “baking”

• I made another backup but didn’t it compress it..

• It clones without problem to 8 & 16GB cards! So the problem seems to be with the unzipping.

• I format the SD card with: Prep for NOOBS

• I start the restore and watch the disk listing in the Finder Window:

  – the SD card unmounts

  – mounts as Raspberry

 – mounts again as Raspberry

So I have two “Raspberries” listed just as AP-B gets stuck (BTW, the red progress text is behind the action. By the time it says “unmounting”, the card has been unmounted for several seconds.)

• Ejecting one of the Raspberries, leaves behind a black-icon “Raspberry” card. It acts as an alias and can be removed from the sidebar by right-clicking on it.

I hope this helps, it looks like a bug because I reproduced it several times with 3 cards.

Tests by Hans:

It’s been quite the busy week, so I apologize for not getting to this earlier.

My 2 new 32Gb microSD cards arrived (SanDisk).

I’ve tested the following;

1) Prep SD for Noobs with ApplePi-Baker,

2) Downloaded the offline version (link), unzipped it on the SD card and booted it in my Raspberry Pi 2.

3) Resized partition and installed Raspbian, tested it, rebooted it twice, to make sure all worked OK.

After that, I tried 5 ways of making a backup to an IMG file.

ApplePi-Baker, and via Termina: “dd if=/dev/diskx” and “dd if=/dev/rdiskx” both with no “bs” parameter (I belief it defaults to bs=512) and once with “bs-512k”. Note that those backups took hours! Not kidding.

This resulted in 5 identical images. 

I compared it with “cmp” (Terminal), and matched their size matches. All files are exactly: 31,104,958,464 Bytes, and “cmp” indicated no differences between the files.

I checked the SD card capacity, and both SD cards show exactly the same size (31,104,958,464 Bytes – Terminal: “diskutil info /dev/diskx”), and that’s the exact same size as the IMG files. So no reason to think this would go wrong …

So, next step, I tried to restore all of these images (totally useless IMO, but I had to make 10000% sure), and they all result in the same scenario; 

Raspberry Pi 2 boots and during the boot sequence some EXT4-fs error occur until it gets to a point where things go super slow, trying to address more EXt4-fs issues. After 10 minutes of waiting, none of finish these “repairs”.

I did try restoring with ApplePi-Baker and the “dd’ variants that I mentioned before, they all result in the same issue.

So I can confirm that, especially with Raspbian, that there are some weird issues.

I’ll do some more testing with for example Win32DiskImager, and some other IMG files like OpenElec, as I noticed interesting things going on with Raspbian. 

I also notice quite a lot of similar complaints, all over the web (not ApplePi-Baker specific), where some even point at the Raspberry Pi firmware, others guess SD card compatibility issues, etc. Now which, if any, is indeed at fault or not, I don’t know. 

I’m beginning with Win32DiskImager – to exclude that there might be a bug in “dd” (highly unlikely).

Possible fix???

I think I might have bumped into a possible fix … I hope. Tested it 3 times now and each time it worked.

So I got a little desperate, trying to find a solution and figured; what if the disk gets ejected by the system either too early and not all data has been written to the SD card? This is where I’d want to use “sync” to force writing possible buffers:

The sync() function forces a write of dirty (modified) buffers in the block buffer cache out to disk. The kernel keeps this information in core to reduce the number of disk I/O transfers required by the system.

I’ll admit that this was a wild guess … but … it seems to work!

So I took the backup I made with ApplePi-Baker in my previous test.

Restored it with ApplePi-Baker and right after that, before doing any unmounting/ejecting, did the following in Terminal (assuming the disk is /dev/disk4):

sync
diskutil unmountDisk /dev/disk4

And after that I removed the disk and booted the Raspberry Pi from it … 

I did see in all 3 tests, a brief “fsck” check popup, fixing some small issues, during Raspbian boot, after which it would reboot once.

However: After that one time reboot, Raspian boots just fine, as it should do.

I’m curious if others could test this theory as well … 

One "funny" thing that I see in the fsck check during the first boot after restoring: fsck says that there are too many blocks, well, actually, 1 to many haha.

I just modified ApplePi-Baker to execute a sync at the end.
I'm also thinking of adding the "auto eject" option after completing a successful flash ...

Tested OpenElec last night.

- Downloaded the IMG and used ApplePi-Baker to flash it
- Booted and had OpenElec resize as it should
- Rebooted to make sure everything worked
- Made a Backup with ApplePi-Baker
- Flash the 2nd SD card

Booted just fine! So ... I'm optimistic that this fixed the problem.

Hans,

With the new version, I was successful in making a working backup of the 32GB card in my system.  Before, I had only been able to do that once and onto a 64gb card.  So it looks promising.

Thanks,

Irv

Thanks for taking the time to test Irv!

And ... great to hear that it seems to work! 

I'll wait a day or two to see if others get a chance to test this version. After that I'll release it.

I did see a comment from Tim (link) that he has to reformat the SD card before flashing - not sure if you're experiencing the same issues.
If so, then I can re-enable the original method in which I repartition/clean the SD card before flashing.

It might be superstitious behavior but ever since my first few attempts, I've been reformatting the SD cards before every backup attempt.

In trying to test the new version yesterday, I made a new .zip backup.  AP-B hung-up at the start of the write attempt (as I've previously reported).  I then tried to expand the zip file.  Just as Stuffit Expander was finishing the file, SE silently failed and quit and deleted it's temp file.  Might there be a buffer involved when you're creating zip files?  Might this buffer also need flushed?

Irv

Crap I forgot about that one ... actually I redirect dd output to zip or 7zip. Let me try adding a sync after that one as well, who know.

As for the superstitious thing; I got a few of those as well ....

I made some more updates for ApplePi-Baker, if you'd like to test.

1) I added a "ApplePi-Baker is still working" indicator, so you can see it is still busy. (left of the title "ApplePi-Baker")
2) Added Sync for restore and backup (for compressed images)
3) Changed status refreshrate 
4) Rewrote the "eject" ad "unmount" procedure
5) Modified the unmount when restoring procedure
6) Fixed a cosmetic bug when the initial ETA was a ridiculous amount of hours

So the mentioned 32Gb backup/restore, and the compressed backup issues for large SD cards should be addressed.

Feel free to give it a try.

The 1.9.1 version is working for me with 8BG cards and .img backups. 

But it's consistently not working with .zip files created with 1.9.1 or earlier versions.  They hang at the very beginning when restoring.

If I take a 1.9.1 .img file and ZIP it with the Mac's built in compression, and restore that using 1.9.1.  It works.  So that's an easy workaround.

So it looks like the problem is with creating .zip files in 1.9.1 and earlier.

-----

I haven't been able to test 32GB cards.  The NOOBS card that came with my Rpi is the largest of the four 32GB cards I own. I ordered a 64GB card to try reducing the partition size of my image but the card is lost in the mail.  I tried ignoring the 1.9.1 warning that my cards aren't big enough but the Rpi hangs up during boot when using them.

Irv

Thanks Irv for taking the time and effort to do some testing.
Seems there still is an issue with Zipping it.
I'll do some investigating - hopefully today - and see what I can find that might cause this issue.
Did you try 7zip as well?

With ZIP I know the image is basically store without a real name (limitation of piping to Zip), 
I also noticed that 7Zip offers better compression.

I didn't try 7zip but I will.  My 64GB card should arrive today.  Hopefully, I'll be able to trim my 32GB image.

Irv

No worries, I'll be patient ... thanks you very very very much for taking the time to test!