I’m very excited to announce that ApplePi-Baker now supports shrinking and expanding of Ext2, Ext3 or Ext4 Linux partitions on a Mac!!
This is something I wasn’t able to do before and is commonly seen as a challenge on a Mac (since MacOS does not support these Linux File Systems).
This update can only shrink or expand Linux partitions on a MBR partitioned disks (very common format used for the Raspberry Pi).
Keep in mind: I’ve done quite a lot of successful tests, but by lack of beta testers, I’m still marking it as experimental.
During backup, if the option has been enabled, ApplePi-Baker will locate a Linux partition and if found shrink it to it’s minimal size, and update the IMG file and it’s partition table (the source will not be affected). This will result in a much more compact IMG file, and offers a restore to a device of a different capacity.
On Restore, this same option will expand that Linux partition to the maximum available size on the target disk.
Obviously, the target device needs to be able to accommodate at least the minimum size of the IMG file.
Since quite a bit needs to be done (find partition, check file system integrity, resize partition, etc), this will naturally slow down the backup and restore process, but it will help restoring to different sized target disks/SD-card.
Not only that, version 2.1.0 is much faster than 2.0.0 – I’ve seen speed increases up to 300% during backup.
You can find the latest version of ApplePi-Baker in the original article (please place comments there).
Comments
There are 50 comments. You can read them below.
You can post your own comments by using the form below, or reply to existing comments by using the "Reply" button.
Hi, I downloaded v2.1.3 and selected the shrink option when making an IMG of my card, I get the proper “Shrink IMG Enabled” log informing me it is creating a *.tmp file, then after an hour or so of reading my 64gb card, I get the following error when the resize process starts after cloning completed:
Resizing – Atempting IMG Shrinking
RESIZE – Attempting to minimize a Linux partition in the in the IMG file “backup.tmp”
RESIZE – ERROR: Source IMG File does not exist
Resize Error – Source IMG File does not exist…
and this I fail to get a proper .img backup..
Any idea what I am doing wrong or need to modify? Attempted 3 times and all of them failed
Thank you!
Antonio Dávila
Hi Antonio,
I’m sorry to hear you’re running into issues.
Unfortunately, I didn’t have many beta testers, and to make things worse; I’m traveling .
First thing I’d check is disk space, and if I recall correctly see if the directory ~/tmp was made.
What is the build version you’re using? (look in the “About”, your can right click the version info to copy it to clipboard and paste it here)
hans
Hi Hans! Thank you very much for your reply, let me check into the disk space issue, seems that could be the problem since I have a 512gb SSD Mac and I was taking the IMG from a 64gb card, will check about the /tmp folder too.
Checked the build and its 2.1.3 build 98.
Just discovered Apple Bi Baker a few days ago and I am in love with it, as soon as I can shrink the file I will be ecstatic to say the least =)
Enjoy your trip, I will get back to you as soon as I am done making some experiments and find out what is causing the problem.
Regards
Antonio
Antonio
Thanks Antonio!
Glad to hear you’re liking ApplePi-Baker.
As for the resizing; I’m considering doing it in a different way, but that will take some work to get that done.
Yeah I think being able to shrink has almost become a requirement to have that when working with Raspberry Pi images and/or SD cards
hans
You know the IT adage/joke that says: have you tried restarting it? hahaha
Well I turned of my computer for the weekend and came back to it this Monday morning, checked what you mentioned and decided to try it one more time with the shrink img option. Let it run for about an hour or so, went to pick some coffee, came back to my mac and… surprise!! There was a 4gb .img waiting for me.. much smaller than 64gb!
I immediately grabbed a 16gb ram SD card, tried to burn the 4gb .img file with balena etcher as I used to do with all raspbian images and… surprise! it is working perfectly!! I am so so so happy an excited for all things I will be able to accomplish now!
I clicked on the donate with Flattr but it did not provide any way to pay, only got to the “donate to” screen. Will find a way to make it work and make sure you receive my donation, ApplePi Baker is awesome!! really awesome! =)
Thank you very very much Hans!
Regards
Antonio
Antonio Davila
Awesome!
Being active in IT myself; yeah, most users are not pleased when you tell them to do that hahaha.
By the way; when you do a restore with APB, it can actually expand the 4Gb to fill the entire 16Gb card.
I’d be happy to hear if the shrinking and expanding works well. It has been tested by myself, with success, but it would be great to hear confirmation from other.
Oh and thanks for reminding me about Flattr. I still have to remove that.
They are not paying out the money people donated, no matter what I try .
PayPal is an option though: hans at luijten dot net.
But … never a requirement. Sharing the link is just as great of a donation
hans
I carefully assume you’re in Mexico?
If so: thanks for the donation (it didn’t mention an email address).
I most certainly will cash that in tonight for a few beers
hans
Yup! Monterrey, Mexico to be precise, so go and enjoy some Coronas!! (not our best beer but our most famous heh).
I am still making images to my heart’s content and backed up all my Pi’s ,thank you again!
Sidenote beer fact: Monterrey is the beer capital of Mexico since 1890, (Corona is from the competitor made in Mexico City), Monterrey’s brewery building was modeled after Heineken’s brewery in Amsterdam which was state of the art back then. Nowadays the buildings are way more similar because they both feature Heineken signs since the Mexican and Dutch companies merged a few years back, who would have expected that? haha
Antonio
Nice! I’ve never been to Mexico – even though I have been “relatively” close (I used to live in Houston).
Yeah Corona with a lime will be good on a sunny day.
But it’s like Dutch beer; Heineken most certainly is not the best either .
It is pretty cool though to have significant history in your backyard
My favorite beers are Belgian, and my favorite pilsners are German though.
And my safe beer is Corona
What would be the best Mexican beer in your opinion? I wouldn’t mind trying to find some in the local store.
hans
Yeah, Houston is “relatively” close to Mexico, but anyway it is still a five and a half hours drive to the border at Laredo, TX.
My fave Mexican beer would be Bohemia and a close second Indio. Funny thing, our own beers are getting harder to locate in convenience stores due to Dutch beer brands replacing them, at least since the Heineken-Cuauhtémoc Merger, now its easier to find Amstel and Heineken everywhere versus Carta Blanca and Indio. I guess its mostly due to marketing.
Try to see if you can find out some Indio, Pacifico or Dos Equis, I like those too and they should be much easier to find over there I guess.
Now I gotta go and finish those Heinekens waiting for me in the fridge, cheers!!
Antonio
Haha, yeah relative indeed. But … I could have gone to Mexico – a 5.5 hour drive isn’t too bad.
I’ll go check the local liquor store and see if I can find those brands!
Marketing does seem to ruin everything, doesn’t it?
Enjoy your Heinekens – cheers!
hans
p.s. I had a few Rolling Rock beers from your donation, sitting outside in the sun.
Not world’s best beer either, but it won’t hit you over the head when sitting in the sun, and it drinks really easy .
hans
Just so you know, I flashed the card again with APB and yes, resizes the filesystem perfectly, thank you!
Regards
Antonio
Antonio Davila
Awesome!
If you run into issues, feel free to let me know here or email me.
I’d be happy to help!
hans
I just installed to empty card NOOBS_v3_1_1 which is based on Debian “buster” and then tried to backup it using ApplePi Baker 2.1.0 (91), Helper tool 1.4b in 10.14.4.
I got this error:
Sandra
Hi Sandra,
thank you for reporting this.
Could it be that your disk is full? (also check ~/tmp)
I seem to have to improve some error handling, which I’ll get to when I get back from vacation.
hans
End result is Raspberry.tmp which is 30.78GB.
There is 2TB free in disk. I tried to find any other big files in disk but did not find anything.
Is it possible to rename this file to .iso?
Note that this latest Raspbian is buster version, which is alot different than previous one so i think this causes it.
Sandra
OK, thanks for getting back with that. So this is a bug I should look into (thanks again for reporting!).
If I recall correctly, you can rename the file to a .IMG file (eg. Raspberry.img).
hans
I renamed it and still fails:
Sandra
You may want to disable the “Expand/Shrink” function before flashing the renamed file.
(5th button from the right should be gray)
hans
All those Function buttons was missing until in changed Theme. I have seen these buttons earlier.
I tried flashing when Expand/Shrink was disabled and it worked.
Sandra
Hi Sandra,
good to hear it finally worked out.
What version of ApplePi-Baker are you using? I thought I had fixed the “missing buttons” issue in the latest version.
hans
I use 2.1.0
Sandra
OK that explains it … the most recent version (at this moment) is 2.1.3 (see the original article).
hans
Both “Download Now” and “The direct download link to the latest version will always be https://www.tweaking4all.com/ApplePi-Baker2.dmg” gives same old 2.1.0 file.
Downloaded filename is correct: ApplePi-Baker-v2.1.3.dmg
Size: 4 048 849 bytes
Sandra
That is interesting … nice catch!
Both links give me 2.1.3 (dmg and application).
I did notice that I may not have updated the version number in the info.plist (an Apple specific info file buried in the app bundle) – so I assume you check the version number through the “Get Info” option in Finder (which indeed shows 2.1.0).
When you start APB it will show a version number in the titlebar of the APB window (2.1.3).
Thank you very much for catching that … I’ll try to update the version info correctly in the next few days.
And rest assured that the version number in the title bar of the APB window is the only always 100% correct version number (it’s generated by the development environment I’m working with). I’ll write a script to automatically update the Info.plist file as well so that “Get Info” will display the correct info (it’s a pain to have to update the version number in multiple places).
hans
I use ApplePi-Baker 2.1.3 (Build 98) and i tried first time expand/shrink.
I shrink Raspbian 16GB sd-card and then tried to expand that image to another 16GB sd-card but your app said:
This selected Disk is too small:
/dev/disk3 (32 GB Generic- USB3.0 CRW -SD) (179,830,784 bytes short)
Shrinked image is 4.5 GB and non shrinked data size surely should fit to my new 16GB card.
I tried it and got error: kernel panic not syncing vfs- unable to mount root fs on unknown-block(179,7)
Any way around this?
Sandra
Hi Sandra,
sorry to hear you’re running into issues.
If I understand you correctly, this would indeed be weird – would you be able to email me the entire content of the log window (when you’re trying to flash the shrunken image to the 16Gb SD)?
I assume the “kernel panic” error is when you boot your Raspberry Pi with the SD card?
hans
ApplePiBaker2 is much appreciated. The Shrink and Expand functions are a MASSIVE time gain on OS X. This rocks.
kenneth himschoot
Thanks Kenneth for taking the time to post a thank-you!
It’s highly appreciated!
hans
Hi, great work !
do you have a version 2 for old OS 10.9 ?
thomas
Hi Thomas,
unfortunately, I do not have 10.9 available for any development or testing
I suppose you are not able to update to the latest (or a newer) macOS (thanks Apple!).
hans
Hello, I’ve been using your ApplePi-Baker program for YEARS with RetroPie images. SUPER excited to see you have the shrink feature now!!! since SD cards even by the SAME EXACT manufacturer (and SAME model/size/specs, even ordered form the same exact vendor – “sold by and shipped by Amazon”) can be DIFFERENT sizes over their production runs… meaning I can’t restore to the card if it’s even a few bytes smaller!
Right now I’m trying to make an image of a 32GB RetroPie SD Card on my Mac running OS 10.14.6, then have it resize to be a smaller size. I have 77.2GB left on my SSD (which I thought would be enough, even if I needed DOUBLE the 32gb SD cards’ size). The process fails, and leaves a 16.39gb .TMP file on my desktop. Tried it 3x times now. Error log:
https://pastebin.com/7RtFxfKR
I’m happy to run any further tests. Have another two Macs to try it on too (one newer, one older), but would have to run it off an external drive since those have even smaller SSD’s. I’ll bookmark this and check back if you have anything to try. Thank you!
Jesse
Hi Jesse,
as a rule of thumb (worse case scenario), I always aim for at least 3x the SD card free disk space.
From what I could see, you’re literally running out of disk space.
From you log I can see that the Linux partition is almost 32Gb (doesn’t mean all space is actually used).
So to make an IMG, you’d need at least 32Gb of disk space.
To shrink the partition, the Linux partition (almost 32Gb) needs to be extracted from the IMG file (after making the initial backup) – we’re needing 64Gb disk space at this point.
This extracted partition is then resized, and then needs to be combined with the other pieces of the original IMG, which potentially would generate another 32Gb file, so we need (worse case) 96Gb of disk space.
These are all worse case scenario numbers of course, but in your case if the partition shrink result is anything larger than 13Gb, you’d be out of disk space.
Not to mention: your OS would like to use some space every now and then as well.
Tip:
– Make the backup to an external disk that has enough space (sure, the internal SSD will be faster). Ditto with a restore when using shrink/expand.
– To Restore an IMG file, you will not be required to make a backup with Shrink first. So any IMG file that has the proper layout can be expanded (MBR, Standard partitions, expanding the first Linux EXT partition).
Hope this helps – and thank you for adding the log file externally, this is very considerate of you
Happy New Year!
Hans
Hi Hans! You were SO right – totally a space thing. I let it run overnight while I slept (to my home network drive, took ~1.5 hours), and it worked! I’ll let you know if I have any issues with restoring and resizing the partition (have a quote-on-quote “identical” Sandisk EVO 32gd card that’s maybe 200kb smaller than Sandisk EVO 32gb I originally imaged – which I could NOT previously restore this original 32gb image onto). I’m off from work for the holidays so I’ll be re-doing 3-4 friends RP3’s with this new image. THANK YOU!
Jesse
Awesome! That’s good news
This is also a good reminder for me to do a little work on APB and it’s helper tool.
A more informative error from APB would have saved yo some time.
Let me know if you run into any other question, and … Happy New Year
Hans
So I finally got around to restoring that backup we’ve been talking about – onto that SAME brand/size card, but the one which was just a little bit smaller than the others. Got a “Read/Write Error while copying Linux partition” error while I was restoring it (think on the expanding part). Log:
Relevant snippet from the debug log:
Jesse
Wanted to add this: So last time making the backup the issue was on my MAIN computer – I didn’t have enough space. So I backed it up to my SERVER computer which has a 10TB drive with plenty of space left and the backup then worked.
I was restoring the backup on my Main computer, since my Server doesn’t have a card reader,
Even though the backup file is on the Server, I’m now wondering if my Main computer (the one doing the backup itself) STILL has to have that 3x-times-the-backup space left on it’s SSD? And that I’m now running out of space, again the same exact issue is popping up?
I just googled this statistic, and on brand new higher-end laptop with a SSD, that SSD’s average size is going to be right around 256GB. This means that most people aren’t going to have enough space to backup even a smaller 32GB microSD card since they’re not going to have 96GB free… unless all this can be done on an external drive…
…so you previously mentioned doing it on an external hard drive… the problem is this network drive is mounted exactly like an external drive would be – no difference to the OS… so it seems like (totally guessing here), that if your program is still setup to ONLY do all this temp-file image resize/extract/etc. on the Main computers internal hard drive no matter what (i.e. there’s no option to do that on the external drive), then this isn’t going to be possible for many people to do.
Anyway, just thought I would mention the above. I’m going to see if I can dig-up a microSD card reader and try it on the server itself, but if I’m right with the above this will only work if the main hard drive with the OS on it has enough space left… which it might not as the server has the majority of it’s storage on the non-main hard drives.
Jesse
Hi JEsse,
I think you stumbled into a little bug in the Restore …
This line tells me that APB is [still] using a temp file on the local drive (versus the disk where the image resides).
19:41:27 Extract IMG – Destination: /Users/___removed my name___/tmp/applepibaker_temporary.img
I will try to work on this next year (2021) – I won’t have time in the next 2 days. Maybe I can add a dialog where APB asks something like
“Would you like the temp files to be stored locally (~/tmp) or where the IMG file resides (/path/to/img)“.
Good catch!
For now: Happy new Year and stay safe and healthy!
Hans
Seems like I can’t comment to your below, but thank you and happy New Years! No worries if it’s not fixed until ‘next year’ – just looking forward to trying it out with the new shrink feature.
Was also wondering if you can take an already existing image, like one I made with the older version of ApplePi-Baker, and shrink that one as well? (I’ve only shrunk on the newer version you just released).
Also, that 32gb image I DID make on the newer version + shrunk, properly expanded on another 32gb card when I did it on my Server which has enough space on the main hard drive left (and seemed to work for the 5 minutes I tried it out on so far).
Jesse
Hi Jesse,
Yes, you can take an existing IMG file and shrink it.
Follow these steps:
1. Open APB, and open the advanced view so you can see all disks.
2. Double click the IMG file in Finder, so it will mount the disk – the “new” disk will now appear in APB as well.
(don’t worry that you will not see all partitions in the Finder).
3. Make a backup in APB of that mounted IMG (drive) with shrink enabled.
This should go very fast (compared to an SD backup).
The result will be a copy of the IMG file, just with shrink applied
I’ll try to find time soon to make a fix for the restore issue
Hans
Crashed twice. Selected the SD card, then went to “Disk to File” and it crashed, twice. Going to reboot that Mac Pro and see if it’ll work again. Apple crash log if it helps:
https://pastebin.com/B3FkTmvU
Jesse
Hi Jesse,
From what I could read form the log;
1) You’re running a rather old macOS version (10.13) – this could be fine though, I just do not have the means to do any testing for this macOS version.
2) Give the new beta 2.2.6 a try (I see you’re running the older 2.2.2 version). In this new beta the issue of running out of disk space has been addressed by making temp files next to the original file (when using shrink or expand).
The new beta may or may not resolve the crash you’re looking at (the widget-set used in my dev tools are updated almost daily and 2.2.6 has been compiled with the latest version).
Hans
p.s. you can find the beta version here.
Hans
hello. i am using this tool to flash an image 128gb to a larger 256gb card. when it was completed, my new SD card had shrunk to 128gb and the leftover space was no longer available. How can i upgrade to a larger card without losing the extra space? i did this so i could have a larger card for games but ended up with a duplicate of what i already had. thank you to anyone who can assist.
cee
Hi Cee!
You can try using the expand option in ApplePi-Baker.
This comes with some requirements though as this only supports standard partitions (not extended partitions), where the first partition is a FAT/FAT32/exFAT partition and the last (if I recall correctly) partition is a Linux Ext partition.
When enabled it will locate that last partition and fill resize it to maximize the use of your SD card.
For some reason some image creators use Extended partitions – not sure what the purpose of that would even be, as standard partitions already allows for 4 partitions (boot, swap, os, data ?). Anyhoo … the advanced info in ApplePi-Baker can reveal some of this.
Another option would be doing this in the OS you’ve flashed on your SD card, but the steps would potentially be rather specific.
There is a RaspberryPi OS (Raspbian?) specific trick, which should be easy, but I honestly haven’t touched a Raspberry Pi since the release of the Raspberry Pi 2. So you may need to search for that one.
This is what I found (source):
Hope this helps … 😊
Hans
Hello Hans. Thank you for the quick reply and information. Please pardon the noob questions as I am new to this but eager to learn. It is possible that I forgot to check the second box in the options before clicking on the file to disk.?? That is the option that is needed correct? Would I also have to check that option when flashing the original img file to the computer? I don’t think I did that the first time around. I am currently flashing the image to disk again but with that second option checked (box second from the left) where it says Options. Thanks for your time my friend
Cee
Hi Cee,
No need to say pardon for asking question
The second icon from the left, in the “Options” section on the right is the one you would need.
Please note that this will only work under the mentioned circumstances (MBR, Standard partition and not extended partitions).
Hans
Thanks again for being so helpful. I’m a little confused on the proper verbiage for my situation. Im attempting to upgrade from a 128gb card to a 256. Each time I have flashed the 128 games card to my new 256….the new card comes out at 128gb… the goal is to flash the original 128gb image onto the bigger card and be able to use that leftover space for more games. Just wanted to make sure we are on the same page. Is this possible with applepibaker?? Your help is much appreciated Hans…
Cee
You’re welcome! And never be afraid to ask – we all had to learn (and we still do) 😉
That is too bad, because Extended Partitions makes resizing unnecessarily more complicated (from a software perspective) while it does not add any value.
If that is the case, then ApplePi-Baker won’t be able to resize that for you. (can you post a link where you got the IMG file from? – I can see if I can help)
Hans