Tuesday, January 25, 2011

Preparing a development environment for the Kobo Wifi

This article is part of an extended series on Kobo development and investigation


Apparently I have to make this more obvious:

  • Take a backup of your Kobo's SD card before doing anything. If something doesn't work, you can't fix your Kobo without this backup. I will not send you a backup.
  • Any device hacking is risky. If you can't afford to break your device, leave now.

First, I'd like to thank the KoboLabs members for being so helpful and responsive as I try to get up and running with Kobo development. They've pushed several tools, config files, scripts etc to the KoboLabs repository, all of which have drastically reduced the amount of time it's taken to get up and running and largely eliminated the need for reverse-engineering work. This is incredibly unusual for an embedded company, and absolutely wonderful. I was already impressed by their ongoing releases of new firmwares for the original Kobo after the release of the Wifi, as most companies just drop supports old products. Now I'm blown away. Please tell people how damn impressive the Kobo folks are with their after-sales product support and maintenance; I know I will be.

In fact, this whole document has been rendered largely unnecessary by improvements to the KoboLabs code drops. They've posted their build scripts and configurations. To get started, just clone the git repo and read documentation/README.

Overview

You will need to download the CodeSourcery toolchain used for Kobo development, then build the libraries the Kobo uses so that you can link your new executables against them. Qt Embedded, with its dependencies, is particularly crucial. The latest GitHub code drop includes the Qt Embedded configuration used on the Kobo, letting you compile a compatible version of Qt.

Installing Sourcery G++

Download Sourcery G++ 2010q1-202 4.4.1 for your platform and run the installer.

If you're on an x64 RPM-based Linux distro, you may need to install a 32-bit userspace to be able to run the tools. x64 Debian/Ubuntu users will have to install them in the 32-bit chroot, and do all future steps described here within that chroot.

cd ~/Downloads
chmod a+x arm-2010q1-202-arm-none-linux-gnueabi.bin
./arm-2010q1-202-arm-none-linux-gnueabi.bin

Accept the default install path of $HOME/CodeSourcery/Sourcery_G++_Lite when prompted. Accept all other installer defaults.

You now need to symlink your tools to new names, as some of the Kobo makefiles etc expect the tools to be named arm-linux-x where they're named arm-linux-gnuegabi-none-x by CodeSourcery. To do this:

cd ~/CodeSourcery/Sourcery_G++_Lite/bin
for f in arm-none-linux-gnueabi-* ; do ln -s $f arm-linux-${f:23}; done

If all is configured properly, after a logout & login or after running "bash -l" you should now be able to type "arm-linux-g++" and get the response "arm-linux-c++: no input files".

Compiling libraries

Now that we have a toolchain in place, including gcc, binutils, glibc, and all of that provided for us by the wonderful folks at CodeSourcery, we can get on with the real work of preparing our dev environment.

Clone a copy of the KoboLabs git repository to get the required sources and patches:

mkdir ~/kobo
cd ~/kobo
git clone git://github.com/kobolabs/Kobo-Reader.git koboreader

This will take a while, as it's downloading the sources to a lot of libraries the Linux kernel used on the Kobo, and more.

Once you've cloned the repository, it should be easy to get going. A reasonable option might be:

echo "DEVICEROOT=$HOME/kobo/fs" > ~/kobo/build/build-config-user.sh
mkdir ~/kobo/tmp
cd ~/kobo/tmp
../koboreader/build/build-all.sh

... which will use the Kobo build scripts to compile all the libraries and install them in DEVICEROOT, which you set to $HOME/kobo/fs . The unpacked sources and temporary object files will be stored in ~/kobo/tmp , so your git clone of the kobo sources doesn't get messed up by the build.

Now you need to build Qt, which you can do by following the instructions in documentation/README.trolltech

10 comments:

  1. First off - thanks for your detailed information on the Kobo. I don't know what I'd have done without it. (Well, probably just leave the device alone, but where's the fun in that?)

    Did you by chance try to compile the new sources, posted in April?
    I do so now, but for some reason or another it doesn't work.
    No doubt it's largely my linux ignorance. I managed to get across several problems (missing programs, old openjade docbook, the IMO wrong instruction in the Kobo doc about QT_EXTRA_ARGS), but right now I'm stumped.

    Apparently there is a problem somewhere at qt.sh, when trying to link libdbus1 . The linker (named as usr/bin/ld ??) says the file is incompatible.

    It seems to me as if the wrong linker is invoked. If I could find where exactly it is called, I'd check right now...

    Any ideas what I may have done wrong, or what I should check?

    ReplyDelete
  2. I haven't rebuilt from the 1.9 sources, no. I pretty much stopped working with the Kobo as an interesting hacking project when I was told that they weren't going to release the qws video driver for the epson eInk screen so it could be used outside the pre-compiled nickel binary.

    The fact that the "plugin api" is completely useless kind of put me off, too. It has *no* visibility into the innards of nickel, no hooks, nothing. Most of Qt's introspection doesn't work with the compile options they've used (to reduce binary size) so I can't even do much in terms of runtime hooking without already knowing what things are called.

    There's plenty of bug fixing I'd like to do, but none of the issues I'm encountering are things I can do anything about without Nickel source code. Since it looks like Kobo won't be releasing the Nickel sources even under NDA or other restrictive terms, there's bugger all I can do.

    I got an Android phone at a similar time, which caused me to completely stop caring about being able to do anything interesting with the Kobo's wifi like slurp news feeds with it. I'd maybe still want do it if the QWS driver were open source or even available as a shared library, but it isn't and isn't going to be.

    The whole thing goes into the "too hard" basket, but it sure was interesting to learn with.

    Maybe if you provided some more specifics - say, put build logs on a pastebin - I could help. I'm not putting too much time into it right now, though.

    ReplyDelete
  3. Thanks for the quick answer.

    That about nickel and the driver is sad. I understand that you'd rather work with something you can actually work with.

    I uploaded the output of the build process onto pastebin: http://pastebin.com/KnfVq3Nu .

    I didn't completely rebuild, I left the tmp directory as is.
    The errors I mentioned at the end (as to be expected, I guess...).

    I'd be glad if you could spare some time and look at it.

    ReplyDelete
  4. Ok, that's weird.

    What does:

    file /home/cyberman/Kobo/KoboReader/lib/libdbus-1.so

    say? How about the output of:

    objdump -x /home/cyberman/Kobo/KoboReader/lib/libdbus-1.so

    ?

    For comparison, please show the output of "file" and "objdump -x" on the copy of zlib in the KoboReader/lib folder.

    Also: is it possible that you've updated or changed your toolchain (compiler etc) or pulled a new copy of the KoboReader repository since you did you first compile? If so, you should delete the working directory ~/Kobo/tmp and re-compile everything from scratch by re-running the build script. The build script isn't smart enough to understand what needs to be recompiled when something changes and what doesn't.

    ReplyDelete
  5. I uploaded the data you requested here: http://pastebin.com/wWrH0gLk

    Hope it is what you meant/need.

    I don't think I've updated my toolchain or anything, but just to make sure I've deleted the old files and restarted compilation.

    ReplyDelete
  6. Hm. I recompiled several times now - even rebuilt the whole system (Virtual machines are a great invention). No luck, same problem.
    I have the nagging feeling that there is something wrong in the build configuration, the same compiler that builds the (x86)tools then tries to link the arm-binaries.

    I better take it to a forum (Mobileread to be precise), perhaps someone else tried it too.

    Thanks for your time.

    ReplyDelete
  7. Cyberman: I haven't tested with 1.9, as I said. Perhaps the Kobo folks changed something in their build that broke it.

    One thing to try is building Qt separately, in case one of the earlier scripts is exporting environment variables that are inappropriate for building Qt.

    You should also check the output of the "env" command to make sure that none of these variables are set before running a build:

    LDFLAGS
    LD
    LDPATH
    LD_LIBRARY_PATH
    LIBRARY_PATH
    CC
    CXX
    GCC
    INCLUDE_PATH
    CPATH

    ReplyDelete
  8. Both libz and libdbus are the same arch with the same flags according to the data you posted. They're both being built as you'd expect and should be compatible with the toolchain.

    As you say, it sounds like it's calling the wrong linker.

    Is it possible your PATH gets messed up so that the x86 `ld' is on the PATH before the arm cross-compiling `ld' ?

    ReplyDelete
  9. I checked the environment variables, nothing that comes even close.

    The path is set up so that the CodeSourcery tools will come first. And as you know, they have their own prefix, so there should be no way they can be accidently called.
    To make sure, I checked the -version output of both. Regular ld is GNU ld fo Debian 2.20.1
    arm-linux-ld is Sourcery G++ 2.19.51

    I'll try to compile QT by itself, as soon as I figure out how. The qt.sh cannot be called alone, but I don't see anything in there that could be a problem.

    My guess is that it's deeper in there, somewhere where the just-in-time compiled tools take over. I *think* it's (at least) one of the makefiles that are created on-the-fly.
    Which would mean one of the .pro or .pri files is wrong. I'll have to invest some time poking around them, I guess...

    ReplyDelete
  10. I have the same problem as Cyberman (/usr/bin/ld: cannot find -ldbus-1)
    Does anyone have a solution for this?

    Thanks.

    ReplyDelete