shabtek

How to clone flash card?
« on: November 09, 2017, 12:07:36 AM »
I want to make a backup
Copy of An sd card that will be bootable in device. Dont know how to format/ clone. I have made a diskimage.dmg but  cannot get it to transfer, not sure about partitions.
"really fine players do not use stomp boxes or master volume, they match the amp to the room and turn it up to 11.  Stevie Ray, BB King, Albert King, Duane Allman, Dicky Betts, Louis Armstrong"
   -CJ


john12ax7

Re: How to clone flash card?
« Reply #1 on: November 09, 2017, 01:12:40 AM »
What OS? On PC there is Rufus a freeware program that makes bootable USB from iso image files, it works well.  Etcher might be an option for Mac.

cyrano

Re: How to clone flash card?
« Reply #2 on: November 09, 2017, 12:18:20 PM »
Etcher might be an option for Mac.

Thx for the tip. Seems like a good alternative to dd. Especially for newbies :D

https://github.com/resin-io/etcher
Why is it people love to believe and hate to know?

shabtek

Re: How to clone flash card?
« Reply #3 on: November 12, 2017, 12:29:34 PM »
Thanks for the tip.
I have been wasting away this weekend trying to get a couple different macbooks up to spec to run etcher...ended up going to mums house, to no avail.

The card i want to clone has several linux partitions and a fat32 partition. It appears on desktop as black rectangle icon rather than white drive/dmg icon.  Needles to say etcher does not want to mount/load it but rather open it to find a file within. 
Does anyone have suggestions how to change the extension or whatever so that etcher will recognize it in its entirety?

Do I need to format the destination in a particular way?

Maybe i'll try formatting with  individual partitions like source and etchering each partition seperately.???
"really fine players do not use stomp boxes or master volume, they match the amp to the room and turn it up to 11.  Stevie Ray, BB King, Albert King, Duane Allman, Dicky Betts, Louis Armstrong"
   -CJ

Re: How to clone flash card?
« Reply #4 on: November 12, 2017, 03:46:10 PM »
Is dd really all that tough to operate? The only odd thing I can think of is choosing the block size and doing some math to see how many blocks you need to copy. Disks used to use 512 byte blocks, but maybe that's changed with modern, large devices. Aside from that, the command is all of 20 characters.

Since you're cloning the whole thing, just copy all of the physical  device's addressable blocks, from block 0 to the max, whether they're used or empty. You'll then get the boot loader, partition table, filesystems, and any possible hidden copy protection stuff - the whole shebang regardless of what it means. No thought required (or desired). Trying to tweezer it apart partition by partition will be needlessly tough, you'll get it wrong, and you don't even need to do this.
« Last Edit: November 12, 2017, 03:57:18 PM by Monte McGuire »

shabtek

Re: How to clone flash card?
« Reply #5 on: November 12, 2017, 07:52:21 PM »
Thanks Monte.  I am not a computer person-- I tried dd before i asked here--but was unsure about what I am doing.
What you said about copy the whole thing makes sense.
Do I need to format or partition the destination or does dd take care of that?
"really fine players do not use stomp boxes or master volume, they match the amp to the room and turn it up to 11.  Stevie Ray, BB King, Albert King, Duane Allman, Dicky Betts, Louis Armstrong"
   -CJ

Re: How to clone flash card?
« Reply #6 on: November 12, 2017, 09:00:57 PM »
Carbon Copy Cloner

Pick the source, pick the target, hit clone.

Re: How to clone flash card?
« Reply #7 on: November 14, 2017, 01:11:35 AM »
Thanks Monte.  I am not a computer person-- I tried dd before i asked here--but was unsure about what I am doing.
What you said about copy the whole thing makes sense.
Do I need to format or partition the destination or does dd take care of that?

The beauty is that you're copying the device block by block, so however it's formatted or partitioned, whatever's in the boot block, all of that, will be copied faithfully by dd. This is why dd is nice - it deals with the source and destination disk as raw blocks, completely uninterpreted, and thus makes a completely faithful copy. You could copy a linux filesystem for that matter - none of this matters.

I’m going into huge detail here, but you need to know what’s going on sufficiently, or else you can do some serious damage with dd.

You're on a Mac, so Disk Utility can help you to identify the devices and see how large they are in their raw state. I'm using El Capitan, so what I write is relative to that, but dd hasn’t changed for decades, and Disk Utility is probably consistent over a number of MacOS versions. I’ll try to condense this, so here we go.

Plug in the source flash drive that you want to copy and then open Disk Utility. On the left side of the Disk Utility window, you’ll see a hierarchical listing of all of the drives in your system, and the partitions from each that the OS has found. The topmost will probably be your boot drive and its partitions (aka volumes), and somewhere below will be your flash device. If the Mac has decided to mount partitions from the flash drive, they will appear below the raw flash device. If any partitions appear below the raw flash device, click on the partition and then press the ‘unmount’ button at the top of the Disk Utility window. Do this for all of the flash partitions so that only the raw flash device is mounted.

Then, click on the raw flash device and press the ‘Info’ button, next to the ‘Unmount’ button. This will bring up a window with a lot of information in it. What you want is the ‘BSD device node’, which will be something like disk2 or something like that.

If you can attach both flash devices at once, don’t remove the source flash, and then plug in the destination flash and see what address it comes up with in the same way. Then, you can copy directly from one device to the other.

Let’s say the source flash is disk2 and the destination flash is disk4. The command you want is:

dd if=/dev/disk2 of=/dev/disk4

That’s it. This assumes a blocksize of 512 bytes, but even if that’s not the case, it won’t matter. It might take longer, but maybe not.

If you can’t have both flash drives on your machine at once, then you need to copy to a file and then copy the file to a drive. This is the command to read the source flash to a file:

dd if=/dev/disk2 of=./flash-contents

This makes a file ‘flash-contents’ in your home directory with the device contents. It’s not a .dmg file, it’s just a file. Delete it after the copy is done.

When that finishes, remove the flash drive and plug in the destination. If the destination device is formatted and the OS mounts volumes, do the unmount thing using Disk Utility and the Info button thing as before. Then use this command, assuming that the destination flash also comes up as disk2:

dd if=./flash-contents of=/dev/disk2

The only gotcha is that the destination flash must be the same size (or greater) than than the source flash drive. Flash media that is ‘8GB’ for example is not actually 8GB, and among all things labeled ‘8GB’ they may be off by a few blocks or more from other ‘8GB’ media. Buy two of exactly the same media, or just verify in Disk Utility with the Info button to make sure that the ‘Volume Capacity’ of the destination is as big or larger than that of the source. You could also just buy a larger size destination - e.g. a 16GB flash to copy an 8GB flash. The extra at the end will just sit there, inaccessible until you reformat the flash.

In theory, every block from the source device will get copied to the destination, and they will appear to be the same drives, with the same formatted capacity, partitioning etc. A block for block clone.

Re: How to clone flash card?
« Reply #8 on: November 14, 2017, 01:21:33 AM »
I forgot one basic step. You type the 'dd' commands into a Terminal window, and you may need to precede them with the word 'sudo', to allow you to have permission to read and write the raw drives. Try it without 'sudo' but if the OS says no, then precede the command with the command sudo, as in "sudo dd if=/dev/disk2 of=./flash-contents"

Also, to delete the temporary file, it may be easiest to use this command after you're done making the clone:

sudo rm -f ./flash-contents

rather than trying to find it through the Finder.

I've been using Unix machines since the early 90s with SunOS 4.x that I forgot to explicitly state this - sorry! The cool part is that a reasonable Unix implementation is inside of a modern Mac, so you can do all of the down and dirty stuff as well as use the nice window and application system.

Read twice, type once, and good luck!
« Last Edit: November 14, 2017, 01:26:25 AM by Monte McGuire »

shabtek

Re: How to clone flash card?
« Reply #9 on: November 14, 2017, 10:53:14 AM »
Thank Monte,bonzai,cyrano

I will post when this gets sorted
"really fine players do not use stomp boxes or master volume, they match the amp to the room and turn it up to 11.  Stevie Ray, BB King, Albert King, Duane Allman, Dicky Betts, Louis Armstrong"
   -CJ


shabtek

Re: How to clone flash card?
« Reply #10 on: November 17, 2017, 12:32:32 PM »
Thank you Monte (guiness)

dd did it, etcher and carbon copy cloner did not,  in my ' dd sudo'attempt before asking here I neglected to unmount disk volumes before attempting to write...dont know if that was also the issue when tryimg etcher and ccc.

Thanks for the step-by-step!
"really fine players do not use stomp boxes or master volume, they match the amp to the room and turn it up to 11.  Stevie Ray, BB King, Albert King, Duane Allman, Dicky Betts, Louis Armstrong"
   -CJ

john12ax7

Re: How to clone flash card?
« Reply #11 on: November 17, 2017, 04:10:29 PM »
Interesting, good you got it resolved.   I might need to try dd in the future.  I've made bootable USB drives,  but trying to clone a windows system drive has never worked for me,  the OS and programs never come out right. Perhaps temporarily mounting in Linux is the answer.

pvision

Re: How to clone flash card?
« Reply #12 on: November 17, 2017, 05:27:15 PM »
Man files in the Terminal are super-useful. Just type man dd to see the BSD General Commands Manual for dd. The same is true for all Terminal comands

man dd <return>

DD(1)                     BSD General Commands Manual                    DD(1)

NAME
     dd -- convert and copy a file

SYNOPSIS
     dd [operands ...]

DESCRIPTION
     The dd utility copies the standard input to the standard output.  Input
     data is read and written in 512-byte blocks.  If input reads are short,
     input from multiple reads are aggregated to form the output block.  When
     finished, dd displays the number of complete and partial input and output
     blocks and truncated input records to the standard error output.

     The following operands are available:

     bs=n     Set both input and output block size to n bytes, superseding the
              ibs and obs operands.  If no conversion values other than
              noerror, notrunc or sync are specified, then each input block is
              copied to the output as a single block without any aggregation
              of short blocks.

etc

You can see attached volumes by typing
 cd / volumes
ls

If you've ever played/fought with DOS then the Terminal is similar with standard commands like cd (change dorectory) and ls (list files)

If you want to know what Sudo does, type man sudo

Nick Froome

john12ax7

Re: How to clone flash card?
« Reply #13 on: November 17, 2017, 06:21:19 PM »
I've been trying out Linux again recently after a long hiatus.  It was able to easily format a USB stick that Windows kept complaining about.

Re: How to clone flash card?
« Reply #14 on: November 20, 2017, 12:47:28 AM »
shabtek - glad it worked out!

Interesting, good you got it resolved.   I might need to try dd in the future.  I've made bootable USB drives,  but trying to clone a windows system drive has never worked for me,  the OS and programs never come out right. Perhaps temporarily mounting in Linux is the answer.

MountIng any of the source filesystems is never a good idea, since the OS can then play with the data, and it's conceivable that your copy won't be internally consistent if the source data changed as it was being copied. If it's unmounted but the device is attached, then it's just a pile of blocks on a device, and the copy will proceed regardless of anything else happening on the computer.

Any machine that has BSD utilities on it will have dd, so OS X is useful for this, as well as Linux. I have to build and then clone some PATA flash drives for a piece of test equipment, so I need a machine that has an old ATA bus on it, and that means an ancient PC with some flavor of Linux. So, it's nice that dd will run on a range of hardware. Linux can handle some seriously crusty old hardware too - stuff like 5.25" floppies and the like, so it's a good swiss army tool to use.

john12ax7

Re: How to clone flash card?
« Reply #15 on: November 20, 2017, 05:25:56 AM »
I've misunderstood then.  I thought you needed to mount something to make it accessible / readable.

pvision

Re: How to clone flash card?
« Reply #16 on: November 20, 2017, 12:15:20 PM »
I've misunderstood then.  I thought you needed to mount something to make it accessible / readable.

To a computer a hard drive is just a collection of blocks of data. To an OS it might be a file system,  a bunch of files, etc. The OS has to mount the drive and make sense of the file system to make sense of the files & folders. But utilities like those we've been discussing don't care about the file system, just the data blocks. They can ignore the subtleties & just copy the blocks. In lots of cases that's a better option. They don't mount the disk - just see it

Probably an imperfect explanation in technical terms but hopefully useful!

Nick Froome

john12ax7

Re: How to clone flash card?
« Reply #17 on: November 20, 2017, 04:58:29 PM »
It's useful. I get it now.  Thanks for the explanation.