Sign up here and you can log into the forum!

Loading custom kernel

Have a question about devices internals, memory layout, reverse engineering, etc---This is the place for anything so technical that it would cause a n00b's head to 'splode

Loading custom kernel   

Postby craigp » Mon Nov 14, 2011 2:22 am

I've read comments on websites and forums about the WDTVlive booting an encrypted kernel and hence we are unable to change it. I've had a poke around and wanted to share my thoughts.

If you take a look on the console when I assume z-boot is doing it's stuff, you will see :

Code: Select all
Processing vmlinux_xload.zbf (start: 0x01740090, size: 0x002baca4)
  Checking zboot file signature .. OK.
  Warning: header version mismatched.
   *** Fully Encrypted.


I've had a look around for the format of .zbf (Z-Boot file Header). It would seem our friends over at opentvixdev have the source for genzbf. From the source you can see there is a 32 byte header made up of 8x 32 bit longs added to the encrypted & compressed kernel. The 2nd long after the magic 'FNIB' is an Attributes flag which specifies if the attached kernel is compressed and/or encrypted. In the case of stock standard Western Digital firmware both the encrypted and compressed flags are set, hence the message *** Fully Encrypted.

Now while I haven't yet had a chance to try, at the moment I can't see why we can't create a compressed kernel (I believe compressed means gzip) but not encrypted and have the bootloader load it, provided we don't set the encrypted flag.

Looking a bit further at the console messages, you will see z-boot copies the contents of vmlinux_xload.zbf out of the romfs in flash and over to DRAM. It then spawns x-loader which decompressed the kernel to 0x84000000 and executes it from 0x84444000. The start address of 0x84444000 is in the .zbf header, but it is also in the elf header of the kernel. If you build the kernel and run readelf -h vmlinux, it will report the entry point address is 0x84444000 which is all consistent.

YAMON allows you to load a kernel via TFTP into DRAM and execute it.

To bring up an ethernet interface:

Code: Select all
YAMON> net init
Ethernet driver for SMP86XX (v1.0)
(MAC 00:90:a9:90:92:91)

em86xx_eth0 - full-duplex mode
em86xx_eth0 - 100 Mbit/s
em86xx_eth0 ethernet start
DHCP was successfully configured.
ipaddr:     192.168.0.50
subnetmask: 255.255.255.0
gateway:    192.168.0.254


Then you will want to load the kernel into DRAM :

Code: Select all
YAMON> load -b tftp://192.168.0.251/vmlinux 0x84000000
About to load tftp://192.168.0.251/vmlinux
Press Ctrl-C to break
................................................................
................................................................
.......................................

Start = 0x84000000, range = (0x84000000,0x84539573), format = BINARY
Length = 0x539574 (5477748)
Checksum = 0x1c67d95a (10'476567898)


And finally execute it :

Code: Select all
YAMON> go 0x84444000 console=ttyS0
em86xx_eth0 ethernet stop


But at this stage, I get nothing. It's like the console is not working properly.

If I try to start execution from a random addresses (e.g. 0x84000000) YAMON throws a reserved instruction exception, so to some degree I reasonably confident something is happening, just the console is not working. Under kernel hacking in the kernel configuration menu, the default kernel command string is console=ttyS0, so I don't think I have to pass any kernel parameters. I've also tried building with registering an early kernel, but no luck yet.

I know b-rad and some other individual's have tried kexec, so that might be my next point of call just to verify the kernel image is fine.

I believe there are some issues with kexec, so hopefully if we load a kernel image (and rootnfs) via YAMON we can verify a kernel and then move onto the next stage of compressing it and adding the .zbf header.

Any thoughts?
craigp
n00b
 
Posts: 3
Joined: Mon Nov 14, 2011 1:41 am

Re: Loading custom kernel   

Postby craigp » Thu Nov 17, 2011 4:57 am

I can now successfully boot my own kernel using YAMON and TFTP. I can mount a rootnfs, but it's having some trouble with init. Wireshark suggest its failing with NFS3ERR_NOENT which means it can't find the file, so I suspect its a problem I have server side with permissions or something - Something to look at for tomorrow.

Once I get over that hurdle, I'll try making an unencrypted .zbf of the kernel and see if I can flash the device.

Code: Select all
**********************************
* YAMON ROM Monitor
* Revision 02.13-SIGMADESIGNS-24-R2.13-24
**********************************
Memory:  code: 0x86000000-0x86060000, 0x85200000-0x85204000
reserved data: 0x86200000-0x86300000, 0x86700000-0x87000000
PCI memory: 0x86300000-0x86700000



NAND FLASH Driver Version [ S I G M  1.0.6 ] on CS 0

!! No NAND hardware found on CS 1 !!


YAMON> net init
Ethernet driver for SMP86XX (v1.0)
(MAC 00:90:a9:90:92:91)

em86xx_eth0 - full-duplex mode
em86xx_eth0 - 100 Mbit/s
em86xx_eth0 ethernet start
DHCP was successfully configured.
ipaddr:     192.168.0.50
subnetmask: 255.255.255.0
gateway:    192.168.0.254

YAMON> load -b tftp://192.168.0.251/vmlinux.bin 0x84000000
About to load tftp://192.168.0.251/vmlinux.bin
Press Ctrl-C to break
................................................................
................................................................
...............

Start = 0x84000000, range = (0x84000000,0x84478085), format = BINARY
Length = 0x478086 (4685958)
Checksum = 0x79e27c84 (10'2044886148)

YAMON> go . "root=/dev/nfs rw nfsroot=192.168.0.251:/home/cpeacock/export/wdtvlivefs,v3,tcp ip=192.168.0.248::192.168.0.254:255.255.255.0:WDTVLive:eth0:off console=ttyS0 mem=200M"
em86xx_eth0 ethernet stop
Linux version 2.6.22.19-19-4 (root@ubuntu) (gcc version 4.3.2 (Sourcery G++ Lite 4.3-51) ) #8 PREEMPT Thu Nov 17 04:32:16 PST 2011
Physical map 0xc0000000 to 0x04000000, max remap/kernel size: 0x0c000000/0x18000000.
Configured for SMP865x, detected SMP8655 (revision unknown).
Detected CPU/System/DSP Frequencies: 499.50/333.00/333.00MHz
SMP86xx Enabled Devices under Linux/XENV 0xcfd0bcbc = 0x001b3efc
PCIHost Ethernet Ethernet1 IR FIP I2CM I2CS USB PCIDev1 PCIDev2 PCIDev3 PCIDev4 SATA SCARD SCARD1
CPU revision is: 0001937c
FPU revision is: 01739300
Determined physical RAM map:
memory: 05000000 @ 04000000 (usable)
Modified physical map 0xc0000000 to 0x04000000, max remap/kernel size: 0x10000000/0x18000000.
User-defined physical RAM map:
memory: 0c800000 @ 04000000 (usable)
Wasting 131072 bytes for tracking 4096 unused pages
Initrd not found or empty - disabling initrd
On node 0 totalpages: 16896
  DMA zone: 33 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 16863 pages, LIFO batch:3
  Normal zone: 0 pages used for memmap
Built 1 zonelists.  Total pages: 16863
Kernel command line: root=/dev/nfs rw nfsroot=192.168.0.251:/home/cpeacock/export/wdtvlivefs,v3,tcp ip=192.168.0.248::192.168.0.254:255.255.255.0:WDTVLive:eth0:off console=ttyS0 mem=200M
Primary instruction cache 32kB, 4-way, physically tagged, linesize 32 bytes.
Primary data cache 32kB, 4-way, physically tagged, no aliases, linesize 32 bytes
Synthesized TLB refill handler (20 instructions).
Synthesized TLB load handler fastpath (32 instructions).
Synthesized TLB store handler fastpath (32 instructions).
Synthesized TLB modify handler fastpath (31 instructions).
Cache parity protection disabled
PID hash table entries: 2048 (order: 11, 8192 bytes)
Using 249.750 MHz high precision timer.
Console: colour dummy device 80x25
Dentry cache hash table entries: 65536 (order: 4, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 3, 131072 bytes)
Memory: 198656k/204800k available (3604k kernel code, 6048k reserved, 763k data, 208k init, 0k highmem)
Calibrating delay loop... 332.59 BogoMIPS (lpj=1662976)
Mount-cache hash table entries: 2048
NET: Registered protocol family 16
PCI: Initializing SMP86xx PCI host controller
PCI: Remapped PCI I/O space 0x58000000 to 0xc0000000, size 64 kB
PCI: Remapped PCI config space 0x50000000 to 0xc0018000, size 10 kB
PCI: Configured SMP86xx as PCI slave with 1024MB PCI memory
PCI: Region size is 131072KB
PCI: Map DMA memory 0x04000000-0x10800000 for PCI at 0x48000000
SCSI subsystem initialized
libata version 2.21 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
NET: Registered protocol family 2
Time: TANGOX clocksource has been installed.
IP route cache hash table entries: 4096 (order: 0, 16384 bytes)
TCP established hash table entries: 16384 (order: 3, 131072 bytes)
TCP bind hash table entries: 16384 (order: 2, 65536 bytes)
TCP: Hash tables configured (established 16384 bind 16384)
TCP reno registered
NTFS driver 2.1.28 [Flags: R/O].
fuse init (API version 7.8)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
tango3dog: Hardware Watchdog Timer for SMP864x/SMP865x 0.1 (def. timeout: 30 sec)
Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0x0 (irq = 9) is a 16550A
serial8250: ttyS1 at MMIO 0x0 (irq = 10) is a 16550A
loop: module loaded
tangox_enet0: detected phy  at address 0x01
tangox_enet0: Ethernet driver for SMP864x/SMP865x internal MAC core 0: 100Mbps Base at 0x26000
tangox_enet0: mac address 00:90:a9:90:92:91
tangox_enet1: unable to autodetect phy
usbcore: registered new interface driver asix
usbcore: registered new interface driver cdc_ether
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx
tangox_bmide: bmide support is disabled
k_name=Tangox SATA 0 driver=Tangox SATA 0
SATA version 0x3139302a ID 0x0 is detected
scsi0 : Tangox SATA 0
ata1: SATA max UDMA/133 cmd 0xa0023000 ctl 0xa0023020 bmdma 0xcdcdcdcd irq 49
ata1: SATA link down (SStatus 0 SControl 300)
k_name=Tangox SATA 1 driver=Tangox SATA 1
SATA version 0x3139302a ID 0x0 is detected
scsi1 : Tangox SATA 0
ata2: SATA max UDMA/133 cmd 0xa0023800 ctl 0xa0023820 bmdma 0xcdcdcdcd irq 62
ata2: SATA link down (SStatus 0 SControl 300)
ohci_hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI) Driver
USB Universal Host Controller Interface driver v3.0
driver tangox-ehci-hcd, 10 Dec 2004
TangoX USB initializing...
tangox-ehci-hcd tangox-ehci-hcd: TangoX USB 2.0
tangox-ehci-hcd tangox-ehci-hcd: new USB bus registered, assigned bus number 1
tangox-ehci-hcd tangox-ehci-hcd: irq 48, io mem 0xa0021400
tangox-ehci-hcd tangox-ehci-hcd: USB 0.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
tangox-ohci-hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI) Driver
TangoX USB was initialized.
Initializing TangoX USB OHCI Controller Membase=0xa0021500, irq=47
tangox-ohci-hcd tangox-ohci-hcd: USB Host Controller
tangox-ohci-hcd tangox-ohci-hcd: new USB bus registered, assigned bus number 2
tangox-ohci-hcd tangox-ohci-hcd: irq 47, io mem 0xa0021500
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
Initializing USB Mass Storage driver...
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
Not supported chip.
mice: PS/2 mouse device common for all mice
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
drivers/hid/usbhid/hid-core.c: v2.6:USB HID core driver
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
ieee80211: 802.11 data/management/control stack, git-1.1.13
ieee80211: Copyright (C) 2004-2005 Intel Corporation <jketreno@linux.intel.com>
ieee80211_crypt: registered algorithm 'NULL'
eth0: link up, 100Mbps, full-duplex, lpa 0xCDE1
IP-Config: Complete:
      device=eth0, addr=192.168.0.248, mask=255.255.255.0, gw=192.168.0.254,
     host=WDTVLive, domain=, nis-domain=(none),
     bootserver=255.255.255.255, rootserver=192.168.0.251, rootpath=
Looking up port of RPC 100003/3 on 192.168.0.251
Looking up port of RPC 100005/3 on 192.168.0.251
VFS: Mounted root (nfs filesystem).
Freeing unused kernel memory: 208k freed
craigp
n00b
 
Posts: 3
Joined: Mon Nov 14, 2011 1:41 am

Re: Loading custom kernel   

Postby recliq » Thu Nov 17, 2011 1:46 pm

looks promising but I'm not really familiar with the wd booting and kernel internals to judge if this is going to work.
But what I know is that if it was that easy i wonder why b-rad hasn't gone this way and is still struggeling with the last bits of getting kexec to work.. ;)

Don't get me wrong, I'm really impressed by what you're doing and I hope you will have success!
It's just that b-rad really knows this device and I would be suprised if you just found a way he didn't think of ;)

Anyways keep it up and keep us posted!! :mrgreen:
­WDLXTV Project Maintainer
-:] If you like my contributions feel free to donate for a beer or a new flash drive. ...and always remember: RTFM! (README, FAQ, WIKI) [:-
User avatar
recliq
WDLXTV Team
 
Posts: 5024
Joined: Thu Apr 15, 2010 8:09 am
Location: Kiel, Germany

Re: Loading custom kernel   

Postby RMerlin » Thu Nov 17, 2011 2:15 pm

recliq wrote:looks promising but I'm not really familiar with the wd booting and kernel internals to judge if this is going to work.
But what I know is that if it was that easy i wonder why b-rad hasn't gone this way and is still struggeling with the last bits of getting kexec to work.. ;)

Don't get me wrong, I'm really impressed by what you're doing and I hope you will have success!
It's just that b-rad really knows this device and I would be suprised if you just found a way he didn't think of ;)

Anyways keep it up and keep us posted!! :mrgreen:


I believe his kexec work was partly done for a customer of his.
WDLXTV Webend maintainer. Visit http://www.lostrealm.ca/wdlxtv to see my other WDLXTV projects.
If you like my work, please consider donating.
User avatar
RMerlin
WDLXTV Team
 
Posts: 3236
Joined: Sat Jun 26, 2010 9:25 am
Location: Montreal, Canada

Re: Loading custom kernel   

Postby recliq » Fri Nov 18, 2011 8:41 am

Now that you mention it, I remember this and think you are right.
­WDLXTV Project Maintainer
-:] If you like my contributions feel free to donate for a beer or a new flash drive. ...and always remember: RTFM! (README, FAQ, WIKI) [:-
User avatar
recliq
WDLXTV Team
 
Posts: 5024
Joined: Thu Apr 15, 2010 8:09 am
Location: Kiel, Germany

Re: Loading custom kernel   

Postby craigp » Sat Nov 19, 2011 4:00 am

Yeah, no luck. Looks like sigma as crippled x-load so it cannot load unencrypted z-boot files.

I normally get execute at 0xXX denied.

Code: Select all
Reading NAND CS0, addr 0x000c0000, size 0x00000800 to 0x01740000
Reading NAND CS0, addr 0x000c0800, size 0x00203000 to 0x01740800
Found.
ROMFS found at 0x0x01740000, Volume name = CRAIG_XLOAD
Found 1 file(s) to be processed in ROMFS.
Processing vmlinux.zbf (start: 0x01740080, size: 0x0020346c)
  Checking zboot file signature .. OK.
  Warning: header version mismatched.
  Execute at 0x84444000 denied.
Done with container 1.


I modified the zbf header of the original WD/sigma encrypted and changed the start address to see what would happen if the load/start address was wrong or simply the kernel wasn't in the correct memory region. When I do this, it hangs with execute at 0x8444400f ..

Code: Select all
Reading NAND CS0, addr 0x003c0000, size 0x00000800 to 0x01740000
Reading NAND CS0, addr 0x003c0800, size 0x002ba800 to 0x01740800
Found.
ROMFS found at 0x0x01740000, Volume name = MIPSLINUX_XLOAD
Found 1 file(s) to be processed in ROMFS.
Processing vmlinux_xload.zbf (start: 0x01740090, size: 0x002baca4)
  Checking zboot file signature .. OK.
  Warning: header version mismatched.
   *** Fully Encrypted.
src_addr = 0x017400b0, dest addr = 0x01f40000
XLOADING src=0x817400b0, dest=0x81f40000, size=0x002bac84
xload.c:77: Waiting for XLOAD completion.
xload.c:87: XLOAD done, status = 0x6.
  Decompressing to 0x84000000 .. OK (5423151/0x52c02f).
  Load time total 0/0 msec.
Execute at 0x8444400f ..


Therefore I would say xload is deliberately throwing the denied message and its not just an incorrectly packaged .zbf file.


I understand the boot up process much better now. It appears xboot/zboot does it's stuff and then based on the Z.default_boot environment variable executes from "Containers" 0 to 2 - this is sigmblocks c, d & e as defined by z.boot0, z.boot1 & z.boot2 environment variables.

sigmblockc contain YAMON, while sigmblockd contains the kernel and sigmblocke contains a shadow kernel.

If xload has trouble loading one of the kernels e.g. below it will switch between the two i.e change the def_boot (short for Z.default_boot).

Code: Select all
Reading NAND CS0, addr 0x000c0000, size 0x00000800 to 0x01740000
Reading NAND CS0, addr 0x000c0800, size 0x002ba800 to 0x01740800
Found.
ROMFS found at 0x0x01740000, Volume name = MIPSLINUX_XLOAD
Found 1 file(s) to be processed in ROMFS.
Processing vmlinux_xload.zbf (start: 0x01740090, size: 0x002baca4)
  Checking zboot file signature .. OK.
  Warning: header version mismatched.
   *** Fully Encrypted.
src_addr = 0x017400b0, dest addr = 0x01f40000
XLOADING src=0x817400b0, dest=0x81f40000, size=0x002bac84
xload.c:77: Waiting for XLOAD completion.
xload.c:87: XLOAD done, status = 0xbb.
XLOAD failed.
Done with container 1.
read def_boot=1, status=6.
going to set def_boot=2, status=6!!


When x-boot loads, you can select which container/boot index to use by pressing the 0, 1 or 2 key. This is why when you press 0, xboot will load YAMON.

YAMON is naturally encrypted and signed as well :

Code: Select all
Reading NAND CS0, addr 0x00080000, size 0x00000800 to 0x01740000
Reading NAND CS0, addr 0x00080800, size 0x0002ec00 to 0x01740800
Found.
ROMFS found at 0x0x01740000, Volume name = YAMON_XLOAD
Found 1 file(s) to be processed in ROMFS.
Processing yamon-xload.zbf (start: 0x01740080, size: 0x0002f154)
  Checking zboot file signature .. OK.
  Warning: header version mismatched.
   *** Fully Encrypted.
src_addr = 0x017400a0, dest addr = 0x01f40000
XLOADING src=0x817400a0, dest=0x81f40000, size=0x0002f134
xload.c:77: Waiting for XLOAD completion.
xload.c:87: XLOAD done, status = 0x6.
  Decompressing to 0x85200000 .. OK (354848/0x56a20).
  Load time total 0/0 msec.
Execute at 0x85200000 ..

CS 0 vendor id 0x2c.......
CS 0 device id 0xda.......
................................................................................................................................................................................................................................................................doing Super block Sanity checks... location 4
doing Managment block Sanity checks ...

CS 1 vendor id 0x00.......
CS 1 device id 0x00.......



**********************************
* YAMON ROM Monitor
* Revision 02.13-SIGMADESIGNS-24-R2.13-24
**********************************
Memory:  code: 0x86000000-0x86060000, 0x85200000-0x85204000
reserved data: 0x86200000-0x86300000, 0x86700000-0x87000000
PCI memory: 0x86300000-0x86700000



NAND FLASH Driver Version [ S I G M  1.0.6 ] on CS 0

!! No NAND hardware found on CS 1 !!


but is capable of spawning unsigned kernels as I have done using tftp.

YAMON like most bootloaders should be able to set environment variables to 'script' start up (variable start), so I'm going to have a go changing the default boot to YAMON, then scripting YAMON to load a unencrypted kernel from FLASH . . . . . I'll see if I can get that working.

I also notice over at http://wiki.opentvix.com/Main_Page they have a Tvixfw applet to decrypt and encrypt firmware although, I suspect the certificates may be different. They also have some information on the X-load Remote Procedure Calls XRPC to the xpu, so there appears to be some work going into this area. I also note one post from someone that took the easy approach - get rid of x-load all together and load your own bootloader first like YAMON.
craigp
n00b
 
Posts: 3
Joined: Mon Nov 14, 2011 1:41 am

Re: Loading custom kernel   

Postby Alicia » Sat Nov 19, 2011 6:16 am

What you are trying to do is way, way over my head but I keep a close eye on this thread.
Very interesting what you are trying to achieve, and that you keep trying, despite all the hurdles WD throws at you.
1.03.01-WDLXTV 0.5.8.1 + Cylent ShowcaseV2 Mod + pure-ftpd + nfs
User avatar
Alicia
DLX'er
 
Posts: 180
Joined: Sun Apr 25, 2010 12:08 pm
Location: the Netherlands

Re: Loading custom kernel   

Postby bschmidt » Sun Dec 11, 2011 4:17 am

Hi there,

is there any progress on this? I highly appreciate your efforts and would be very happy to be able booting a vanilla kernel.

Thanks!
Bernd
bschmidt
n00b
 
Posts: 12
Joined: Wed Feb 09, 2011 9:13 am

Re: Loading custom kernel   

Postby craig » Tue Dec 20, 2011 12:27 am

I haven't sorry Bernd. It will be a Christmas Holiday job - its the silly season at the moment, and I haven't been able to get back to it.

The last I got was the YAMON bootloader spawning automatically at boot. I can't quite work out where YAMON is storing it's environment variables - it's probably been crippled :)
craig
n00b
 
Posts: 1
Joined: Tue Dec 20, 2011 12:22 am

Re: Loading custom kernel   

Postby RMerlin » Tue Dec 20, 2011 7:06 am

craig wrote:I haven't sorry Bernd. It will be a Christmas Holiday job - its the silly season at the moment, and I haven't been able to get back to it.

The last I got was the YAMON bootloader spawning automatically at boot. I can't quite work out where YAMON is storing it's environment variables - it's probably been crippled :)


LIES. WD doesn't cripple, they merely "enhance". Or at least that's the word their friends in the movie industry told them to use.

<cough>
WDLXTV Webend maintainer. Visit http://www.lostrealm.ca/wdlxtv to see my other WDLXTV projects.
If you like my work, please consider donating.
User avatar
RMerlin
WDLXTV Team
 
Posts: 3236
Joined: Sat Jun 26, 2010 9:25 am
Location: Montreal, Canada

Next

Return to WDTV Live

Who is online

Users browsing this forum: No registered users and 1 guest