Donnerstag, 7. Januar 2021

Debian on WD My Cloud Home single-bay (MCH) - part 2

 In part 1 Debian Buster was installed on the Western Digital My Cloud Home. Unfortunately the default Kernel lacks quite a bit of features. Compiling our own kernel is desirable. Before we getting started to compile away, some base understanding of the device's boot process is required.

Boot considerations

 When booting into Rescue mode (pressing the Reset button on Power-up) the system seems to look for the following files on an external USB stick:

  • bluecore.audio (some sort of firmware for the RTD1295)
  • sata.uImage (Kernel)
  • rescue.sata.dtb (Kernel device tree)
  • rescue.root.sata.cpio.gz_pad.img (Root-fs - padded to 4096kBytes)

The original software runs Android. That's the reason  why you'll find 24+ partitions on the harddisk:

parted -l
Model: ATA WDC WD80EFAX-68L (scsi)
Disk /dev/sataa: 8002GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start      End           Size          File system     Name         Flags
 1      34s        2047s         2014s                         FW_TABLE     msftdata
 2      2048s      67583s        65536s                        KERNEL_A     msftdata
 3      67584s     133119s       65536s                        ROOTFS_A     msftdata
 4      133120s    198655s       65536s                        ROOTFS_B     msftdata
 5      198656s    200703s       2048s                         FDT_A        msftdata
 6      200704s    202751s       2048s                         FDT_B        msftdata
 7      202752s    210943s       8192s                         AFW_A        msftdata
 8      210944s    276479s       65536s                        KERNEL_B     msftdata
 9      276480s    342015s       65536s                        ROOTFS_GOLD  msftdata
10      342016s    344063s       2048s                         FDT_GOLD     msftdata
11      344064s    352255s       8192s                         AFW_B        msftdata
12      352256s    354303s       2048s                         BOOTCODE32   msftdata
13      354304s    356351s       2048s                         BOOTCODE64   msftdata
14      356352s    358399s       2048s                         BL31         msftdata
15      358400s    360447s       2048s                         BL32         msftdata
16      360448s    425983s       65536s                        KERNEL_GOLD  msftdata
17      425984s    434175s       8192s                         AFW_GOLD     msftdata
18      434176s    499711s       65536s        fat32           CONFIG       msftdata
19      499712s    2138111s      1638400s      ext4            SYSTEM_A     msftdata
20      2138112s   3776511s      1638400s      ext4            SYSTEM_B     msftdata
21      3776512s   5414911s      1638400s      ext4            CACHE        msftdata
22      5414912s   9609215s      4194304s      ext4            DATA         msftdata
23      9609216s   13803519s     4194304s      linux-swap(v1)  SWAP         msftdata
24      13803520s  15628053163s  15614249644s  ext4            DISKVOLUME1  msftdata
If you ran the original install from the Russian site, your bootConfig on partition 18 looks something like this:
2:B:2:;

The first column is the BOOT_STATE, the second the A or B type partitions, and the last is the number of boot attempts. Further reading on this can be done in the bootloader (cmd_boot.c) source code supplied by Western Digital. Since the install leaves most partitions untouched, setting the first to 5 would probably perform a factory reset if you ever wanted to go back (haven't tried that!!!)

For anything permanent we should therefore choose the B marked partitions, also occasionally refernced as Rescue

The first partition is somewhat special as it contains information for the bootloader from which sector to load firmware, kernel, and dtb. I believe uBoot would be able to operate on files in partitions, which would've been more hacker-friendly - in this case we have to supply a valid firmware table (map). To make matters worse there is some checksum over each referenced partition included in the map data. Hence every time a modification to a partition is made the FW_TABLE needs updating.

Besides using the obscure Russian fwtutil-1d it is probably wiser to use Silvio Gissi's fwtablectl. Here the first partition is decoded as follows:

Firmware 1: GoldKernel RO:true Compressed:false Version: 0 Size: 12727296 (12727296 with padding) Disk Offset: 184549376 (sector 360448) Load Address: 0x03000000 Checksum: 0x4b610726
Firmware 2: GoldRescueDeviceTree RO:true Compressed:false Version: 0 Size: 61199 (61440 with padding) Disk Offset: 175112192 (sector 342016) Load Address: 0x01f00000 Checksum: 0x001eb5dc
Firmware 3: GoldRescueRootfs RO:true Compressed:false Version: 0 Size: 12582912 (12582912 with padding) Disk Offset: 141557760 (sector 276480) Load Address: 0x02200000 Checksum: 0x3fff93a7
Firmware 4: GoldAudio RO:true Compressed:false Version: 0 Size: 3243056 (3243520 with padding) Disk Offset: 218103808 (sector 425984) Load Address: 0x01b00000 Checksum: 0x0dd71a22
Firmware 5: uBoot RO:true Compressed:false Version: 0 Size: 0 (0 with padding) Disk Offset: 181403648 (sector 354304) Load Address: 0x00000000 Checksum: 0x00000000
Firmware 6: Kernel RO:true Compressed:false Version: 0 Size: 12710400 (12710400 with padding) Disk Offset: 1048576 (sector 2048) Load Address: 0x03000000 Checksum: 0x4b571cf5
Firmware 7: RescueDeviceTree RO:true Compressed:false Version: 0 Size: 62778 (62976 with padding) Disk Offset: 102760448 (sector 200704) Load Address: 0x01f00000 Checksum: 0x001ff85e
Firmware 8: KernelDeviceTree RO:true Compressed:false Version: 0 Size: 62778 (62976 with padding) Disk Offset: 101711872 (sector 198656) Load Address: 0x01f00000 Checksum: 0x001ff85e
Firmware 9: RescueRootFS RO:true Compressed:false Version: 0 Size: 4194304 (4194304 with padding) Disk Offset: 68157440 (sector 133120) Load Address: 0x02200000 Checksum: 0x0df89573
Firmware 10: KernelRootFS RO:true Compressed:false Version: 0 Size: 4194304 (4194304 with padding) Disk Offset: 34603008 (sector 67584) Load Address: 0x02200000 Checksum: 0x14169514
Firmware 11: Audio RO:true Compressed:false Version: 0 Size: 3243056 (3243520 with padding) Disk Offset: 103809024 (sector 202752) Load Address: 0x01b00000 Checksum: 0x0dd71a22
Firmware 12: RescueAudio RO:true Compressed:false Version: 0 Size: 3243056 (3243520 with padding) Disk Offset: 176160768 (sector 344064) Load Address: 0x01b00000 Checksum: 0x0dd71a22
Firmware 13: RescueKernel RO:true Compressed:false Version: 0 Size: 8983912 (8984064 with padding) Disk Offset: 108003328 (sector 210944) Load Address: 0x03000000 Checksum: 0x37f0c764

Building the Kernel

In the same WD source package referenced above, a working 4.1.17 Kernel tree can be found at linux-kernel/linux-kernel/. I did not try to use the included compiler for cross-compilation but compiled it on the MCH instead. To get it to compile I had to install an old gcc-4.8 from Jessie

echo "deb http://archive.debian.org/debian jessie main contrib non-free" > /etc/apt/sources.list.d/jessie.list
apt-get update && apt-get install gcc-4.8
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 60 \
 --slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-4.8 \
 --slave /usr/bin/gcc-nm gcc-nm /usr/bin/gcc-nm-4.8 \
 --slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-4.8

Some components of the provided Kernel are very verbose by default - namely the RTC driver and the network driver. This patch comments a bunch of verbose outputs.

I also created a .config file which includes all modules to get Docker running along with NFS+CIFS filesystem modules.

The build process is no different than any other Kernel compile:

make menuconfig
make -j4 Image
make -j4 modules dtbs DTC_FLAGS="-p 8192"
make modules_install

Before a Kernel is made permanent in the boot FW_TABLE it is advisable to copy everything to a USB stick and see if the Kernel actually boots. (e.g. using the original install USB stick without!!! the OMV directory as we don't want to re-install)

cp arch/arm64/boot/Image /dev/sda1/sata.uImage
cp arch/arm64/boot/dts/realtek/wd-monarch-1GB.SATA.dtb /dev/sda1/rescue.sata.dtb

NOTE: in some cases I had to unplug the device for the boot to succeed - a reboot didn't get the device up! I Believe having seen some similar remarks in the community forum.

Installing the Kernel

To permanently install the Kernel it has to be copied to the appropriate partition along with the device tree config. The above mentioned fwtablectl will then fix the checksums in the FW_TABLE.

dd if=arch/arm64/boot/Image of=/dev/sataa8
dd if=arch/arm64/boot/dts/realtek/wd-monarch-1GB.SATA.dtb of=/dev/sataa6
fwtablectl-arm64 firmware update /dev/sataa1 RescueKernel arch/arm64/boot/Image
fwtablectl-arm64 firmware update /dev/sataa1 RescueDeviceTree arch/arm64/boot/dts/realtek/wd-monarch-1GB.SATA.dtb

Known Limitations

  • As of now I didn't get the PWM device to work that controls the LED
See my next post for how to control the LED.

20 Kommentare:

  1. Hello,

    Can we contact to talk about wd cloud? I'm trying to change kernel on wd cloud to newer one and add lvm. Need advice. Could You help me with this subject?

    Regards,

    AntwortenLöschen
    Antworten
    1. Honestly I just took the Kernel, WD provided... there have been quite some changes which are not in upstream.

      Let me know where I can try to help!

      Löschen
  2. Hi, can you please write a better tutorial for compiling the kernel (using a clean debian system) or publish your compiled kernel. I tried compiling the kernel for a few hours, but always failed.

    AntwortenLöschen
    Antworten
    1. The steps above were actually used to compile on a "clean" Debian Buster. Key is to install gcc-4.8 from Jessie. My compiled kernel is available here: https://loetzimmer.de/patches/

      Löschen
    2. Thanks, I also tried compiling on a clean debian with gcc 4.8, but it always fails with
      gcc: error: unrecognized command line option '-mlittle-endian'
      gcc: error: unrecognized command line option '-mgeneral-regs-only'

      Löschen
    3. Maybe I should've been more clear in my post, I did not cross-compile but compiled on the MCH instead.

      Löschen
  3. Great article!
    Just a novice-question: What benefits does this kernel have over the original? Are the 24+ partitions gone?
    Thanks!

    AntwortenLöschen
    Antworten
    1. Compiling your own kernel will allow you to use NFS, CIFS, Docker.

      The partition layout is more or less a requirement of the boot loader. Unless you compile and flash your own Uboot you'll have to stick to the 24 partitions.

      Löschen
    2. CAn i put your kernel (https://nerdprojekte.wordpress.com/2021/02/22/wd-my-cloud-home-to-linux-server-3-installing-a-new-kernel/) in the fox debian 9 image (https://fox-exe.ru/WDMyCloud/WDMyCloud-Home/Debian/)?

      Löschen
    3. Sure, go ahead! (It's not MY kernel at all ;) )

      Löschen
  4. Thanks for you work!
    It would be very helpful for less experienced users if your kernel would be part of the fox debian 9 image.

    AntwortenLöschen
  5. It is possible to add the virtualization so that we can use the qemu-kvm? i am able to install qemu and start windows xp but without kvm is really slow

    AntwortenLöschen
    Antworten
    1. check https://loetzimmer.de/patches/ there is an image and modules with "_virt" in the name. Try if that works for you!

      Löschen
    2. Unfortunately is not working, i put it and after the nas didn't start anymore, after fixed and installed qemu again i receive this message:

      "kvm" accelerator not found.
      No accelerator found!"

      Löschen
    3. I tried to install also aqemu because i use xfce and when i select in VM accelelrator KVM then IBM PC 64bit, i receive the error KVM KERNEL MODULE NOT LOADED

      Löschen
    4. Hello. i noticed now that in the file "MCH_modules_4.1.17 virt" there is not the directory KVM.

      Löschen
    5. Hello! I am following this subject, I downloaded the virt version too some week ago and I have this problem, I receive this error too "KVM KERNEL MODULE NOT LOADED", on the link https://downloads.wdc.com/gpl/GPL_MCH_6.6.1-123_20191114.tar.xz in the directory kernel there are those module, but it looks like are not applied on the file "MCH_modules_4.1.17_virt.tgz", i am trying to compile following the guide but I keep having errors starting already from:
      make -j4 Image
      make -j4 modules dtbs DTC_FLAGS="-p 8192"
      make modules_install
      I was able to apply a usb3 to pci-e and have other ports like wifi and vga, I installed Qemu and windows XP but unfortunately is using only one core, limited by the missing module, ok the performances are not like a normal pc, but is usefull when you can use the MCH like a substitute to the normal pc, for example now I am able to stop using the computer for converting video and audio for work using the FFMPEG on the MCH, this hardware can make things that are incredible, is slow but perfect for many other things

      Löschen
    6. Wow relly good article, compliments!!
      Please, I am becoming crazy to compile the kernel so I am able to have this module, but i receive error too, in the module archive there is not the directory virt/kvm...kvm.ko, I am not able to put this file inside the module and make it so I can have the KVM working with qemu, is there some way to make it work? I would like to use this function so I can emulate other OS and make some tests.

      Please Help!!!

      Löschen
  6. Please, can you add the rtl8822bu wifi driver to the kernel (it is in GPL the kernel/phoenix/src/drivers/wifi)?

    AntwortenLöschen
    Antworten
    1. clarification - the driver is located here:
      \GPL_MCH_6.6.1-123_20191114\linux-kernel\phoenix\system\src\drivers\wifi\rtl8822bu\

      I redesigned everything into one partition /dev/sataa20 25GB,
      https://ds-blobs-3.cdn.devapps.ru/23944179/%C1%E5%E7%FB%EC%FF%ED%ED%FB%E9.jpg
      if you can add this driver for common devices it will be fantastic!

      Löschen