Sign up here and you can log into the forum!

unpack / repack a firmware

Have a question about devices internals, memory layout, reverse engineering, etc---This is the place for anything so technical that it would cause a n00b's head to 'splode

unpack / repack a firmware   

Postby b-rad.cc » Sun May 16, 2010 8:46 pm

This is a simple guide to how to unpack / repack a firmware.

First a little history...

On the original WDTV and other similar devices before it kernel page size was a standard 4KB. 4KB is used as kernel page size in most modern systems, and therefore standard cramfs tools worked just fine with the firmware images. This all changed however when Sigma released their 2.0 SDK, which changed the kernel page size to 16KB...only thing is they didn't publish this little factoid due to their extreme security policies. Now when the WDTV Live came out it used the 2.0 SDK...and when I got my hands on a device when it hit the market I instantly hit a roadblock trying to port over wdlxtv-live. Well luckily I don't give up very easily so I got to reverse engineering. It was really a mystery at first, because while cramfs images mounted and were read fine on the device itself--those same images were invalid off the device. Was their some sort of encryption at play here? I couldn't see how their could be, unless it applied specifically to cramfs as all other filesystem images mounted correctly on the device. So what did I do? I spent pretty much the next 100hours, straight, mapping out the cramfs file structure by hand. Talk about the hard way. What did I find? Well cramfs is a sort've dictionary based filesystem, you have all the filenames listed at the start of the filesystem and each one has an address and an offset. The address is basically the block number and the offset is where the filecontents start. Note: completely laymanizing the description. ;) Hmmm...all still looked good and valid, like cramfs is supposed to be. So then I re-generated an exact copy of an official fw rip using the normal cramfs tools and compared it byte-by-tedious-byte trying to find where they differed. This is where I noticed something peculiar, in the new firmware images there were only a quarter of the filesystem blocks! Eureka. Once I had discovered this and re-looked at the cramfs source code it was pretty obvious that kernel page size was really the only variable option--and it even says right in the source code AND in the accompanying NOTES file that block size is equal to kernel page size. Maybe I need to learn to read before I go about things the hard way. :ugeek: So now all that was needed was a bit of confirmation before I became the first wdtvl rma. Well heres how I confirmed it:

Code: Select all
cat /proc/meminfo | grep Mapped
cat /proc/vmstat | grep nr_map


Divide the two and you get ~16KB page size, which coincidentally :roll: is block size used in cramfs. After that it was a piece of cake and cramfsck only required an increase in the rom buffer size to accomodate larger block size. At this point I flashed wdlxtv-live-0.1 and promptly went to sleep :lol:

Ok, I'll shut up now--just figured I might as well finally make public exactly how I figured it all out ;)

firmware structure is as follows:
Code: Select all
[32 Byte md5sum][fw cramfs image][16 Byte signature]


the 32 Byte md5sum is the last part of the firmware generated. The signature details you can find here.

Heres the run down:

You have a wdtvlive.bin (WDTV Live) or wdtvhd.bin (WDTV G2) in front of you.
Code: Select all
tail -c +33 wdtvlive.bin > wdtvlive.stripped.bin
cramfsck-16k -x dump wdtvlive.stripped.bin


There, simple as that.

Now in the opposite direction.
Code: Select all
mkcramfs-16k dump/ newFW.bin
signFW newFW.bin signature
cat newFW.bin signature > newFWx.bin
md5sum newFWx.bin | head -c 32 > wdtvlive.bin
cat newFWx.bin >> wdtvlive.bin
rm newFW.bin newFWx.bin


Voila, ez pz lemon cheezie.

Now if you're rebuilding a firmware and have modified files do not forget to update the modified files md5sums in /md5sum.txt.

Attached are the binaries & source files modified to 16k block size. Compile the source on your own to produce binaries or use the actual 32bit binaries I use, its your choice.
Attachments
cramfs-16k-binaries.tgz
cramfs-16k binaries
(31.05 KiB) Downloaded 2883 times
cramfs-1.1-16k.tgz
16k block size cramfs tools source code
(25 KiB) Downloaded 1041 times
PM's are for private matters only, please post public matters on the forum to help others who might have the same issue.
:mrgreen:
User avatar
b-rad.cc
WDLXTV Team
 
Posts: 3003
Joined: Sat Apr 03, 2010 9:35 am
Location: New York

Re: unpack / repack a firmware   

Postby simplytb » Mon May 17, 2010 12:32 pm

Awesome investigation job!!!
Thanks for sharing.

SimplyTB
simplytb
WDTVer
 
Posts: 20
Joined: Wed Apr 21, 2010 2:37 am

Re: unpack / repack a firmware   

Postby juliojs » Mon May 17, 2010 2:33 pm

Thank you!

I'll re-check my repack process... ;)
Image Android app-> WDTV MediaPlayers Remote
WDTV blog (Spanish)-> wdtv-osdmod.blogspot.com
juliojs
OSD Themer
 
Posts: 87
Joined: Fri Apr 16, 2010 1:01 am

Re: unpack / repack a firmware   

Postby murlock » Tue May 18, 2010 7:25 am

Please note that a clone of WD TV HD requires to update /conf_src/version otherwise after flashing custom firmware, boot on it doesn't occurs.
murlock
n00b
 
Posts: 3
Joined: Tue May 11, 2010 10:13 pm

Re: unpack / repack a firmware   

Postby b-rad.cc » Tue May 18, 2010 7:59 am

its not required to modify /conf_src/version at all.
PM's are for private matters only, please post public matters on the forum to help others who might have the same issue.
:mrgreen:
User avatar
b-rad.cc
WDLXTV Team
 
Posts: 3003
Joined: Sat Apr 03, 2010 9:35 am
Location: New York

Re: unpack / repack a firmware   

Postby tjay » Fri May 21, 2010 5:14 am

thanks for sharing brad!

and now a little script that handles all the steps ;)
Code: Select all
wdtv:~# ./wdtvfw_tool
usage: ./wdtvfw_tool <pack|unpack> <firmware folder> <firmware>
wdtv:~# wdtvfw_tool unpack ./rootfs ./wdtvlive.bin
stripping firmware ...
unpack cramfs image ...
unpacked to ./rootfs
wdtv:~# wdtvfw_tool pack ./rootfs ./wdtvlive.new.bin
create cramfs image ...
sign image ...
./wdtvlive.new.bin created
wdtv:~# md5sum *.bin
61b5b33ac11a04e92635d823f0ffcf19  wdtvlive.bin
61b5b33ac11a04e92635d823f0ffcf19  wdtvlive.new.bin


with "md5sum.txt"-creation you get different final md5sums so I left this step out for demonstration

wdtvfw_tool:
Code: Select all
#!/bin/sh

cram_missing () { echo -e "please make mkcramfs-16k and cramfsck-16k available to your PATH\ndownload it here http://forum.wdlxtv.com/viewtopic.php?f=43&t=537" && quit; }
usage () { echo "usage: $0 <pack|unpack> <firmware folder> <firmware>" && quit; }
quit () { rm -f $tmp && exit 1; }

tmp=`tempfile`
type -P cramfsck-16k &>/dev/null || cram_missing
type -P mkcramfs-16k &>/dev/null || cram_missing


[ "$1" == "" ] || [ "$2" == "" ] || [ "$3" == "" ] && usage

if [ $1 == "pack" ]; then

  #pack

  [ -e $3 ] && echo "firmware \"$3\" already exists" && quit
  [ ! -d $2 ] && echo "firmware folder \"$3\" does not exist" && quit

  echo "create md5sum.txt ..."
  cwd=`pwd` && cd $2 && find  -not -name "md5sum.txt" -type f -exec md5sum '{}' ';' > md5sum.txt && cd $cwd

  echo "create cramfs image ..."
  mkcramfs-16k $2 $tmp > /dev/null || quit

  echo "sign image ..."
  filesize=`printf %x \`stat -c %s $tmp\``
  [ ${#filesize} -eq 7 ] && filesize="0$filesize"
  sig=""
  for i in 6 4 2 0 ; do
    sig="$sig\x${filesize:$i:2}"
  done
  sig="$sig\x00\x00\x00\x00"

  echo -en "\xCE\xFA\xBE\xBA\x02\x00\x00\x00" >> $tmp
  echo -en $sig | head -c 8 >> $tmp

  md5sum $tmp | head -c 32 > $3
  cat $tmp >> $3
  echo "$3 created"

elif [ $1 == "unpack"  ]; then

#unpack

[ -e $2 ] && echo "firmware folder \"$2\" already exists" && quit
echo "stripping firmware ... "
dd if=$3 skip=2 count=`expr \`stat -c %s $3\` / 16 - 3` ibs=16 obs=4096 of=$tmp &> /dev/null
echo "unpack cramfs image ... "
cramfsck-16k -x $2 $tmp > /dev/null || quit
echo "unpacked to $2"

else

echo usage

fi

quit
tjay
n00b
 
Posts: 3
Joined: Fri May 21, 2010 4:30 am

Re: unpack / repack a firmware   

Postby thespecialist » Sun May 23, 2010 5:25 am

I just wanted to add, if you, like me, modify a file, flash it, try it, modify, flash, try etc, you might get fed up with editing that md5sum.txt file containing all the md5sums. In that case you can just simply delete the specific entry(ies) from that list ! Not all files are on the list and there's no check (and hence no NEED) at all for them to be on the list.

I think this list is *only* used while flashing anyway (?). However, it brings a (very slight) risk to the table of course and it probably is good practice to not do this when you want to release a certain version to the public.
thespecialist
Developer
 
Posts: 410
Joined: Sat Apr 24, 2010 6:59 am

Re: unpack / repack a firmware   

Postby b-rad.cc » Sun May 23, 2010 6:07 am

no, its checked every boot and things can get very interesting trying to recover a wdtv with failed checksums if the proper files get corrupted.
PM's are for private matters only, please post public matters on the forum to help others who might have the same issue.
:mrgreen:
User avatar
b-rad.cc
WDLXTV Team
 
Posts: 3003
Joined: Sat Apr 03, 2010 9:35 am
Location: New York

Re: unpack / repack a firmware   

Postby thespecialist » Sun May 23, 2010 7:29 am

b-rad.cc wrote:no, its checked every boot and things can get very interesting trying to recover a wdtv with failed checksums if the proper files get corrupted.

Hehe, ok, so deleting from the list probably isnt a very good idea ;)
thespecialist
Developer
 
Posts: 410
Joined: Sat Apr 24, 2010 6:59 am

Re: unpack / repack a firmware   

Postby g_krassa » Fri Jun 18, 2010 7:34 am

Hey guys

Can this method be used to unpack and repack a wdlxtv firmare. Basically I am interested in repacking brad's 4.2 firmware with a different OSD and to add some custom plugins to the UMSP directory. I was able to unpack repack the the EXT firmware and I am now interested in creating a flash version.

Any guidance would be great to avoid bricking my wdtv.

Thanks
Last edited by g_krassa on Fri Jun 18, 2010 12:57 pm, edited 1 time in total.
g_krassa
n00b
 
Posts: 8
Joined: Fri Apr 16, 2010 4:04 pm

Next

Return to WDTV Live

Who is online

Users browsing this forum: No registered users and 5 guests

cron