tag:blogger.com,1999:blog-11357017578028053752024-02-20T13:40:05.691+01:00Working with Linuxpela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.comBlogger20125tag:blogger.com,1999:blog-1135701757802805375.post-21314758599710214472011-02-11T18:13:00.000+01:002011-02-11T18:13:53.880+01:00Wide-character API of glibc<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I've had to write an application that deals with text in virtually any language, at least, any latin-based language. The source of the text is in files that are encoded in UTF-8, so the question was "how do I deal with this new type of text data?".</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Glibc (and others) provide the wide char (wchar_t) data type, which is able to represent any character by using a fixed-width representation of a character, in my platform, 32 bits wide.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">There are functions used to manipulate text data in wide char format, and most of them are equivalent to the classic 8-bit string functions: fwprintf, swprintf, wcslen, fputwc, fgetwc, etc...</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">However, there some tricks that must be taken into account. There is a new operator "L" used to generate wide-char text constants, so the statement:</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<div style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"><b>char mystring[] = "hello";</b></span></div><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">in the wide-char form must be expresed as:</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-size: x-small;"><b><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">wchar_t mystring[] = L"hello";</span></b></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Note that the "format" argument used in all <b>wprintf</b> functions is also a wide-char text, so it needs to be converted to wide-char text using the "L" operator as well.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Once the data is in wchar format, all operations are similar to the classic equivalents, but now the point is how to convert to/from the classic string format. Well, to be able to convert to a wchar character which has more bits than the standard 8-bit representation, some kind of encoding must be used. For example, UTF-8 or UTF-16 are methods to represent wide chars using 8-bit octets. It is important to note that the type of encoding used is independent of the code or the functions used in the code. All the code needs are a couple of functions to convert to/from byte sequences from/to wide chars. The particular type of encoding is not considered here and depends on the operating system's resources and their configuration, usually in the internationalization settings.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">For example, the <b>mbsrtowcs</b> and <b>wcsrtombs</b> functions are used to convert from multibyte sequences to wide char strings, and from wide char strings to multibyte sequences, respectively. </span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The setlocale function must be used to select a particular internationalizaton setting used to convert between wide-char and multibyte sequence strings, for example:</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-size: x-small;"><b><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">setlocale(LC_CTYPE, "en_US.UTF-8");</span></b></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">This setting is used by glibc to do the conversion in the mbs/wc functions. Note that the proper conversion files are required. In Linux, these are under the <b>/usr/lib/locale</b> directory, organized by locale name. In the example above, the files searched would be, in precedence order:</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-size: x-small;"><b><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">/usr/lib/locale/en_US.UTF-8/LC_CTYPE</span></b></span><br />
<span style="font-size: x-small;"><b><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">/usr/lib/locale/en_US.utf8/LC_CTYPE</span></b></span><br />
<span style="font-size: x-small;"><b><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">/usr/lib/locale/en_US/LC_CTYPE</span></b></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The file must be generated with the 'localedef' utility, provded by the glibc installation:</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-size: x-small;"><b><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">localedef -i en_US -f UTF-8 /tmp</span></b></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">generates the internationalization files for the locale in the exmaple under the /tmp directory. Embedded system developers should note that the LC_CTYPE file is 256 KB big!, which is too much for some systems. The problem is that, without these files, the mbs/wc and fput/fget functions simply don't work, indicating that "an invalid byte sequence cannot be converted".</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">In this case, one has to write his own conversion functions at the price of losing some portability of the code. For example, it is cheaper in terms of disk space, to write our own UTF-8 to/from wide-char conversion functions, but we are limited to this type of encoding.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-5889077094466386192011-02-11T10:18:00.000+01:002011-02-11T10:18:43.687+01:00Generation of simple PDF files<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The goal is to produce a very simple PDF file that contains unformatted text without using very heavy resources, such as PDF converters, graphics libraries, etc, and match the limited resources available on an embedded platform.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">PDF generation in this simple way is quite straightforward. The Adobe's PDF specification provides a couple of examples of simple PDF files that can be used as a template for a very simple PDF rendering library.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Basically, a PDF file is a collection of objects linked from a global object table (xref) that lists the offsets where each object can be located in the file. Some objects have references to other objects as well.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Problems come when the text to be converted to PDF contains special characters, such as those used in the eastern european ones. The PDF specification states that all characters in the text must have 8 bits only, and PDF doesn't know anything about UTF nor Unicode encodings. For each 8-bit character in the text, a reader will fetch the corresponding glyph from the font file selected in the font-type PDF object. The fetch is done using the 8-bit value of the character and taking into account the encoding of the font. Only two types of font encodings are supported : WinAnsiEncoding and MacExpertEncoding, which are used in Windows and MAC, respectively. This encoding is defined in a "Font" PDF object, or in an "Encoding" object referenced by a "Font" object. However, the windows font encoding is always fixed to the CP-1252 code table. Unfortunately, this encoding only covers some non-ASCII characters, but most of them are not. For example, most of the chars used in eastern european languages (polish, hungarian, etc) are not defined in CP-1252, but in CP-1250 code page.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">With these limitations, it is not possible to use non-CP1252 chars in a text stream of a PDF file. There is, however, a workaround that may work in some cases. PDF provides the so-called "font encoding differences", which is an optional entry of the font encoding dictionary that allows the writer to map a given set of character codes in the text to a set of glyphs in the selected font. Glyphs are defined by a standard name, and a list of official glyph names can be retrieved from the Adobe Glyph name list. However, not all glyphs are available in all fonts, of course.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">This is an example of this mapping:</span><br />
<br />
<div style="font-family: "Courier New",Courier,monospace;"><b>10 0 obj<br />
<<br />
/Subtype /Type1<br />
/Name /F1<br />
/BaseFont /MyriadPro<br />
/Encoding 11 0 R<br />
>><br />
endobj<br />
11 0 obj<br />
<<br />
/BaseEncoding /WinAnsiEncoding<br />
/Differences [ 156 /sacute 230 /cacute<br />
241 /nacute 179 /lslash<br />
]<br />
>><br />
endobj</b></div><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">In this example, there are two objects: a Font object (10) and an Encoding object (11). The font object uses the Encoding object to define the encoding of the font, which in turn includes the encoding differences. In this example, char code 156 is mapped to the <b>"ś" </b>character, code 230 is mapped to <b>"ć"</b>, code 241 is mapped to <b>"ń"</b> and code 179 is mapped to <b>"ł"</b>, which are some of the characters of the polish alphabet, and the codes are taken from the CP-1250 code table, which is NOT supported by PDF.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Note that the font type used (MyriadPro in this case) must contain the glyphs used in the Encoding object: sacute, cacute, nacute and lslash. Otherwise, the .<b>notdef</b> glyph would be displayed by the reader. For exmaple, the basic "Courier", "Arial" and "Helvetica" fonts I tried, did NOT contain these glyphs. In particular, I couldn't find any standard proportional font containing these glyphs.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">So, for this method to work, it requires that the operating system where the PDF reader is running, provides the font defined in the PDF file, which is a non-standard font. A better solution would be embedding these fonts in the PDF file, but this is not a easy task. </span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Unfortunately, this seems to be only safe solution, that is guaranteed to work in any system or environment.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The next step is to understand and learn how to embed font programs in PDF files, in an easy way.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br />
</span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-13946253013776439712011-02-11T09:38:00.000+01:002011-02-11T09:38:47.917+01:00USB File Storage Gadget<span style="font-family: arial;">Today I have enabled the USB gadget support for file storage. The intention is to be able to export files via the USB device interface to a PC.<br />
<br />
The file storage gadget must be enabled at the kernel config menu:<br />
<span style="font-size: 85%;"><span style="font-weight: bold;">USB support -> Support for USB gadgets -> File-backed storage gadget</span><br />
</span><br />
Note that only one USB gadget may be enabled at the same time. If multiple gadgets must be supported, all of them must be configured as modules, so I had to remove built-in support for ethernet gadget from the kernel. Switching the USB function requires removing and installing the proper modules.<br />
<br />
The module for file storage is g_file_storage.o and is installed this way:<br />
<span style="font-size: 85%;"><span style="font-weight: bold;">insmod g_file_storage.o file=/results.bin stall=0<br />
<span style="font-size: 100%;"><span style="font-weight: bold;"><span style="font-weight: bold;"><br />
</span></span></span></span><span style="font-size: 100%;"><span style="font-size: small;">The 'stall' argument is necessary for the USB disk to be properly detected by windows. Linux does not require this argument and the drive can be mounted without problem. If 'stall' is not set to zero and the gadget is connected to a windows PC, the following messages appear:</span><br />
<span style="font-size: 85%;"><span style="font-weight: bold;">g_file_storage pxa2xx_udc: full speed config #1</span><br />
<span style="font-weight: bold;">udc: pxa2xx_ep_disable, ep1in-bulk not enabled</span><br />
<span style="font-weight: bold;">udc: pxa2xx_ep_disable, ep2out-bulk not enabled</span><br />
<span style="font-weight: bold;">udc: USB reset</span><br />
<span style="font-weight: bold;">udc: USB reset</span><br />
</span><br />
<span style="font-size: small;">repeating every few seconds.<br />
<br />
For the 'stall' option to be available, it is necessary to enable the 'file-backed storage gadget in test mode' option in the kernel configuration.<br />
<br />
Multiple files may be specified when the gadget module is installed, thus creating multiple drives visible to the remote host.</span></span><span style="font-weight: bold;"><span style="font-size: 100%;"><span style="font-weight: bold;"></span></span></span></span><br />
<br />
Any volume size may be created but it seems that Windows assigns floppy drive letters if the volume size is similar to a floppy device size. I have tested 720KB and 1440KB only.<br />
<br />
The volume may be declared as read-only by using the "ro=1" parameter at the insmod.<br />
<br />
The backend file may be either a disk partition or a file image.<br />
An initial filesystem image can be created this way:<br />
<br />
<span style="font-size: 85%;"><span style="font-weight: bold;"># dd if=/dev/zero of=results.bin bs=512 count=2880</span><br />
<span style="font-weight: bold;"># mkdosfs results.bin</span><br />
</span><br />
Then, loop-mount the image file and populate the filsystem. Here is where problems came: if a process writes a new file to the loop filesystem, the host side of the USB connection (where the file browser runs) does not see the new file, even if the file browser is refreshed. The only workaround is to unplug/plug again the USB cable. This happens even if a 'sync' command is run on the tester device.</span><br />
<span style="font-family: arial;"><br />
</span><br />
<span style="font-family: arial;">Also, some inconsistences happen if the USB host side writes to the device. The device doesn't see the new files, and vice-versa.</span><br />
<span style="font-family: arial;"><br />
</span><br />
<span style="font-family: arial;">In conclusion, it is quite a good method to export only file from a Linux device, but with some limitations on "live" filesystems.</span><br />
<span style="font-family: arial;"><br />
</span><br />
<span style="font-family: arial;"></span><br />
<span style="font-family: arial;"></span><br />
<span style="font-family: arial;"></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com11tag:blogger.com,1999:blog-1135701757802805375.post-3134032460638859332010-02-03T16:45:00.002+01:002010-02-03T17:01:05.153+01:00Multiple gateways on the same host<span style="font-family: arial;">Having two gateways on the same host, where some processes send outgoing</span> <span style="font-family: arial;">traffic over one gateway while the rest use the other gateway, requires a</span><span style="font-family: arial;"> virtual network interface to be set up, and have a separate routing table</span><span style="font-family: arial;"> so that all traffic to/from this virtual interface uses the secondary</span><br /><span style="font-family: arial;">routing table where the other gateway is in the default route.<br /><br />First, I'll describe the steps one by one and later I'll explain how to make this setup persistent so that the system boots correctly the next time. Let's assume we have two gateways:<br /><br />gw1 : 172.26.2.100<br />gw2 : 172.26.3.100<br /><br /><span style="font-family: arial;"></span>Create a virtual interface which will be used by processes that need to send traffic to gateway gw2:</span><br /><br /><span style="font-family: arial; font-weight: bold;">$ ifconfig eth0:1 172.26.3.209</span><br /><span style="font-family: arial;"></span><br /><span style="font-family: arial;">Create a definition and give a name to the new routing table (index 1, name 'test'):</span><br /><span style="font-family: arial; font-weight: bold;">$ echo "1 test" >> /etc/iproute2/rt_tables</span><br /><br /><span style="font-family: arial;">Show the main routing table</span>:<br /><br /><span style="font-family: arial; font-weight: bold;">$ ip route show table main</span><br /><span style="font-family: arial; font-weight: bold;">172.26.0.0/16 dev eth0 proto kernel scope link src 172.26.3.206</span><br /><span style="font-family: arial; font-weight: bold;">169.254.0.0/16 dev eth0 scope link metric 1002</span><br /><span style="font-family: arial; font-weight: bold;">default via 172.26.2.100 dev eth0</span><br /><br /><span style="font-family: arial;">Clear the secondary routing table:</span><br /><br /><span style="font-family: arial; font-weight: bold;">$ ip route flush table test</span><br /><br /><span style="font-family: arial;">Copy all rules from main table to secondary table, but the default gateway</span><br /><br /><span style="font-family: arial; font-weight: bold;">$ ip route show table main | egrep -Ev "^default" | while read route; do</span><br /><span style="font-family: arial; font-weight: bold;">ip route add table test $route</span><br /><span style="font-family: arial; font-weight: bold;">done</span><br /><br /><span style="font-family: arial;">Add the gateway for the secondary routing table:</span><br /><br /><span style="font-family: arial; font-weight: bold;">$ ip route add table test default via 172.26.3.100</span><br /><br /><span style="font-family: arial;">List the secondary routing table:</span><br /><br /><span style="font-family: arial; font-weight: bold;">$ ip route show table test</span><br /><span style="font-family: arial; font-weight: bold;">172.26.0.0/16 dev eth0 proto kernel scope link src 172.26.3.206</span><br /><span style="font-family: arial; font-weight: bold;">169.254.0.0/16 dev eth0 scope link metric 1002</span><br /><span style="font-family: arial; font-weight: bold;">default via 172.26.3.100 dev eth0</span><br /><br /><span style="font-family: arial;">Add a rule so that for any packet to/from the virtual interface, the</span> <span style="font-family: arial;">secondary routing table is applied:</span><br /><br /><span style="font-family: arial; font-weight: bold;">$ ip rule add from 172.267.3.209 lookup test</span><br /><span style="font-family: arial; font-weight: bold;">$ ip rule add to 172.267.3.209 lookup test</span><br /><br /><span style="font-family: arial;">At this point, traffic originated from interface eth0:1 will use gateway gw2 (172.26.3.100) and traffic from interface eth0 enroutes via the default gateway gw1 (172.26.2.100). Try and compare:<br /><br /><span style="font-weight: bold;">$ traceroute -s 172.26.3.206 www.google.com</span><br /></span><span style="font-family: arial; font-weight: bold;">$ traceroute -s 172.26.3.209 www.google.com<br /></span><br /><span style="font-family: arial;">In order to make the changes above persistent, edit/create the following files:<br /><br />1. Add a description for the 'test' routing table (already done):</span><br /><span style="font-family: arial; font-weight: bold;">$ echo "1 test" >> /etc/iproute2/rt_tables</span><br /><br /><span style="font-family: arial;">2. Create file '/etc/sysconfig/network-scripts/ifcfg-eth0:1' containing the</span><br /><span style="font-family: arial;">configuration of the virtual interface:</span><br /><br /><span style="font-family: arial; font-weight: bold;">DEVICE=eth0:1</span><br /><span style="font-family: arial; font-weight: bold;">ONBOOT=yes</span><br /><span style="font-family: arial; font-weight: bold;">SEARCH="mydomain.biz"</span><br /><span style="font-family: arial; font-weight: bold;">DOMAIN="mydomain.biz"</span><br /><span style="font-family: arial; font-weight: bold;">DNS1=172.26.2.200</span><br /><span style="font-family: arial; font-weight: bold;">DNS2=172.26.2.201</span><br /><span style="font-family: arial; font-weight: bold;">BOOTPROTO=none</span><br /><span style="font-family: arial; font-weight: bold;">NETMASK=255.255.0.0</span><br /><span style="font-family: arial; font-weight: bold;">IPADDR=172.26.3.209</span><br /><span style="font-family: arial; font-weight: bold;">TYPE=Ethernet</span><br /><span style="font-family: arial; font-weight: bold;">USERCTL=no</span><br /><span style="font-family: arial; font-weight: bold;">PEERDNS=yes</span><br /><span style="font-family: arial; font-weight: bold;">IPV6INIT=no</span><br /><span style="font-family: arial; font-weight: bold;">NM_CONTROLLED=no</span><br /><br /><span style="font-family: arial;">3. Create file '/etc/sysconfig/network-scripts/route-eth0:1' containing the</span> <span style="font-family: arial;">'test' routing table:</span><br /><br /><span style="font-family: arial; font-weight: bold;">table test 172.26.0.0/16 dev eth0 proto kernel scope link src 172.26.3.206</span><br /><span style="font-family: arial; font-weight: bold;">table test 169.254.0.0/16 dev eth0 scope link metric 1002</span><br /><span style="font-family: arial; font-weight: bold;">table test default via 172.26.3.100</span><br /><br /><span style="font-family: arial;">4. Create file '/etc/sysconfig/network-scripts/rule-eth0:1' containing the</span> <span style="font-family: arial;">rules for the virtual interface:</span><br /><br /><span style="font-family: arial; font-weight: bold;">from 172.26.3.209 lookup test</span><br /><span style="font-family: arial; font-weight: bold;">to 172.26.3.209 lookup test</span><br /><br /><span style="font-family: arial;">The above explanations have been tested on a Fedora 10 distribution.</span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com3tag:blogger.com,1999:blog-1135701757802805375.post-31739453859564989992010-01-15T16:45:00.003+01:002010-01-15T16:58:34.766+01:00Extracting the contents of a ramdisk image<span style="font-family: arial;">Sometimes it is necessary to examine the contents of a ramdisk image (initrd). An initrd image is basically a gzip-compressed cpio archive.<br /><br />Here are the steps used to extract the files contained in a ramdisk:<br /><br /><span style="font-weight: bold;">gunzip initrd</span><br /><span style="font-weight: bold;">mkdir tmp</span><br /><span style="font-weight: bold;">cd tmp</span><br /><span style="font-weight: bold;">cpio -i </span><br /><br />Similarly, to build an initial ramdisk image from a directory:<br /><br /><span style="font-weight: bold;">cd tmp</span><br /><span style="font-weight: bold;">find . | cpio -o -H newc | gzip -9 ../initrd.img</span><br /><br />Where 'newc' is the name of the format used in the cpio archive.<br /><br />In a Fedora distribution, the ramdisk init script is a nash shell-script. Nash is a very reduced footprint shell with built-in commands targetted to ramdisk operations such as device node creation, module loading, root device creation, root pivot, etc.<br /><br />One of the most important nash commands is 'mkrootdev', which creates the root device specified as an argument. After this command is run, the "mount /sysroot" command is executed, which mounts the root filesystem on the root device.<br /><br />If the root device is not correctly specified here, the following error will show up:<br /><br /><span style="font-style: italic;">"mount: error mounting /dev/root on /sysroot as ext3: No such file or</span><br /><span style="font-style: italic;">directory"</span><br /><br /><span style="font-weight: bold;">mkinitrd</span> is the responsible for creating the init script contained in a initrd image, and the name of the root device is obtained from the <span style="font-weight: bold;">/etc/fstab</span> file, when the kernel rpm is installed (not when it is built!). So, it is important to have a correct fstab file in the filesystem before the kernel is installed, otherwise mkinitrd won't be able to figure out the root device and set the correct arguments to the mkrootdev command.<br /><br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-71269374457165623632010-01-14T11:26:00.004+01:002010-01-15T14:27:16.850+01:00Add a new config patch to a Linux kernel RPM<span style="font-family:arial;">Sometimes it is necessary to patch a kernel and rebuild the original rpm so that the installation of the new kernel and its modules is easier. I will not explain here how to patch a kernel, so I assume that we already have a patch available that has been tested and applies correctly.<br /><br />We must have the source rpm of the kernel we intend to patch and rebuild. The following explanation has been tested on a kernel rpm from a Fedora 10 distro and, in particular, I will focus on patching the kernel config files rather than patching the source files.<br /><br />First of all, an explanation of how kernel configuration is achieved by the spec file of a Fedora kernel rpm:<br /><br />1. Some extra config files are provided along with the kernel source tarball. These files are named 'config-*' and are declared in the spec file using the 'sourceNN:' directives. The config files are hierchical, that is, the definitions add hierchichally one on top the other. For example, the configuration file for the i686 architecture is produced by adding the <span style="font-weight: bold;">config-generic</span>, <span style="font-weight: bold;">config-x86-generic</span> and <span style="font-weight: bold;">config-i686</span> config files. The merge.pl perl script, also provided as an extra source file to the rpm, is responsible for doing the merge of the config files.<br /><br />2. The config-* files are copied into the buildroot directory of the RPM, that is under <span style="font-weight: bold;">BUILD/kernel-<version></version></span> directory.<br /><br />3. A 'make configs' rule is executed so that the config-* files are merged and produce a set of kernel-*.config files also in the build root directory. All configs are generated, even if they are not intended for the architecture that is going to be built.<br /><br />4. At this point, all patches are applied using the 'ApplyPatch' macro in the spec file. Note that any patch that changes the kernel configuration must change the kernel*.config files, Patching the config-* files would not work as these are not taken into account for the build process after the patches are applied. Similarly, patching the .config file is not an option since this file will be overwritten by each kernel-*.config file during the build process.<br /><br />5. The config files not intended for the architecture that is being built are deleted.<br /><br />6. 'make oldconfig' is run on all the remaining kernel-*.config files, and the resulting config file is saved in the configs/ directory and removed from the root build directory. This is why the kernel-*.config files are not present even if we only run the patch stage of rpmbuild (-bp).<br /><br />So, to have a patch that changes kernel config we have two options:<br /><br />A. Edit the SOURCE/config-* files provided by the kernel source RPM and rebuild the source rpm with the new ones.<br /><br />B. Generate a patch that modifies the kernel*.config files.<br /><br />There some pros and cons to each option. Option B is more modular, so in case we choose to remove some kernel configuration in the future, all we have to do is remove the patch from the spec file, whereas if we use option A, the original source file is modified and it makes more difficult to revert some config changes.<br /><br />On the contrary, option A is easier than option B because the kernel*.config files that must be patched are autogenerated during the rpm build process and are not available before then.<br /><br />In order to apply option B, we should have a pristine kernel build root containing the patched kernel and the kernel*.config files so that these can be patched. Then, make a copy of the entire root and edit the kernel config files for the intended architectures, generate a patch by diff'ing both directories, copy it into the SOURCE directory and declare it in the SPEC file. Here is an example:<br /><br />1. Generate a pristine build root of the linux kernel rpm<br /><br /><span style="font-weight: bold;">$ rpm -i kernel-2.6.27.5.src.rpm</span><br /><span style="font-weight: bold;">$ cd ~/rpmbuild</span><br /><br />Edit the SPEC file, by adding an 'exit 1' sentence after the last patch is applied (a call to the 'ApplyPatch' macro). This causes the patch process to be interrupted after the kernel*.config files are merged but before they are oldconfig'ed and moved to the configs/ directory. Perhaps there is a more elegant way to do this, but I don't know of it.<br /><br />Extract the sources and apply the patches:<br /><br /><span style="font-weight: bold;">$ rpmbuild --target=i686 -bp SPECS/kernel.spec</span><br /><br />Note the -bp option tells rpmbuild to stop after the source tarball is extracted and all patches are applied. Because we added the 'exit 1' statement, rpmbuild will return an error which of course can be ignored.<br /><br />2. Make the changes in the kernel config files<br /><br /><span style="font-weight: bold;">$ cd BUILD/kernel-2.6.27.5</span><br /><span style="font-weight: bold;">$ cp -a linux-2.6.27.5.i686 linux-2.6.27.5.i686.new</span><br /><span style="font-weight: bold;">$ cd linux-2.6.27.5.i686.new</span><br /><br />Edit each kernel-*.config files that applies to the intended architectures.<br /><br />Changing the config files may be a bit painful, depending on the dependencies of the config items that we change. To make sure that the changes are ok, at the end we must run '<span style="font-weight: bold;">make nonint_oldconfig</span>' with does some sort of check on our config file. Another option is to copy each kernel*.config file as .config and configure the kernel manually using '<span style="font-weight: bold;">make menuconfig</span>', and then copying it back to the original file.<br /><br />3. Generate the patch<br /><br /><span style="font-weight: bold;">$ cd ..</span><br /><span style="font-weight: bold;">$ diff -urN linux-2.6.27.5.i686/ linux-2.6.27.5.i686.new/ > newpatch</span><br /><br />Edit newpatch and make sure that only the changes to the kernel-*.config files are included. Then, rename and move the new patch to the SOURCES directory:<br /><br /><span style="font-weight: bold;">$ mv newpatch ../../SOURCES/linux-2.6.27.5-newconfig.patch</span><br /><br />And declare the new patch in the SPEC file, by adding this line after the last call to the ApplyPatch macro:<br /><br /><span style="font-weight: bold;">ApplyPatch linux-2.6.27.5-newconfig.patch</span><br /><br /><span style="font-weight: bold;"># END OF PATCH APPLICATIONS</span><br /><br />And, of course, remove the 'exit 1' statement.<br /><br />4. Rebuild the RPM<br /><br />As usual, run:<br /><br /><span style="font-weight: bold;">$ rpmbuild -ba --target=i686 SPECS/kernel.spec</span><br /><br />After these steps, both a binary RPM and a source RPM will be present under the RPMS/i686 and SRPMS directories, respectively.<br /><br />Of course, if we already have a patch for the kernel, all we have do is copy it to the SOURCES directory with a proper name, add it to the spec file by using the ApplyPatch macro, and rebuild the rpm (rpmbuild -ba).<br /><br />Note the Fedora kernel spec file builds several flavours of the same kernel architecture, for instance: with or without PAE, debug, SMP, and several combinations of these. So many combinations may take a lot of time to build the rpms. If we are interested in only the base kernel without PAE, debug, etc, all we have to do is add the "<span style="font-weight: bold;">--with baseonly</span>" option to the rpmbuild command. Other variants are allowed, --with / --without pae, debug, etc....<br /><br /><br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-22217969517349617312009-07-06T16:31:00.003+02:002009-07-06T16:45:11.017+02:00Some problems with GPS and NTPD<span style="font-family: arial;">I found some problems when trying to set up the ntpd server connected to a GPS.<br /><br />First, I had to solve how to create the devices required by ntpd: /dev/gps0 and /dev/gpspps0. This is easy: just create a udev rules file, for example /etc/udev/rules.d/90-gps.rules:<br /><br /><span style="font-size:85%;"><span style="font-weight: bold;">SUBSYSTEM=="pps", MODE="0660" GROUP="uucp" SYMLINK="gps%k"</span><br /><span style="font-weight: bold;">KERNEL=="ttyS0", SYMLINK="gps0"</span><br /></span><br />the above creates the following links:<br />gps0 -> ttyS0<br />gpspps0 -> pps0<br /><br />and the target devices are accessible to members of the uucp group. Therefore, the 'ntp' user must belong to the uucp group:<br /><br /><span style="font-size:85%;"><span style="font-weight: bold;"># usermod -G ntp,uucp ntp</span><br /></span><br />Then ntpd experienced some serious jitter problems in the system clock, and I found out that the ntpd driver was not using the PPS signal, but only the NMEA output. Running ntpd in debug mode (-d flag) showed a 'permission denied' error when opening /dev/gpspps0. The file permissions are ok, but the problem was in the selinux layer. I inadvertently had the selinux enabled. Disabling it fixed the problem (set to 'disable' in /etc/selinux/config).<br /><br />Also, the serial port had to be carefully set up before ntpd was started, otherwise, some interactions in the tty driver would be executed, and the DCD interrupt detection would be lost, as explained in the previous posting. The minimum settings of the serial port at start-up are:<br /><br /><span style="font-size:85%;"><span style="font-weight: bold;"># stty -F /dev/gps0 raw ispeed 4800 ospeed 4800 -hupcl</span><br /></span><br />In particular, finding the selinux problem was quite confusing sometimes. If ntpd is started from the command-line as root, the "permission denied" error does not show up. However, if started using the 'service ntpd start' command, the error appears.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-30961349441331599702009-07-01T12:35:00.002+02:002009-07-01T12:54:15.295+02:00Synchronizing an NTP server to a GPS/PPS<span style="font-family: arial;"><span style="font-family: arial;">The goal now is to have a Garmin GPS 18LVC driving a PPS (pulse-per-second)</span> <span style="font-family: arial;">signal to the ntpd server for a highly accurate time reference server.</span><br /><br /></span><span style="font-family: arial; font-weight: bold;"><span style="font-family: arial;">1. The GPS device</span><br /> </span><br /><span style="font-family: arial;"><span style="font-family: arial;">There are at least a couple of ways to propagate the PPS signal to the</span> <span style="font-family: arial;">ntpd server, plus some variants in each case. However, the GPS device must</span> <span style="font-family: arial;">be seen as a device that sources two different types of data: the absolute </span><span style="font-family: arial;">date and time, and the 1-Hz clock signal (PPS).</span><br /><br /><span style="font-family: arial;">The first one provides the complete information about when it is now,</span> <span style="font-family: arial;">including a complete time and date, but with poor accuracy because </span><span style="font-family: arial;">this information is sent over the data lines of the serial port and encoded </span><span style="font-family: arial;">using some type of protocol, i.e. NMEA. PPS provides a very accurate clock</span><br /><span style="font-family: arial;">(about 1 uS in the GPS 18LVC device) but without any reference to the</span> <span style="font-family: arial;">absolute time. This clock is wired to the DCD (Data Carrier Detect) pin of </span><span style="font-family: arial;">the serial port. In other words, PPS tells us with good precision when each </span><span style="font-family: arial;">second begins, but it doesn't tell us which second it is. This timing</span><br /><span style="font-family: arial;">information must be combined with the protocol messages sent by the GPS to</span> <span style="font-family: arial;">have both precision and a complete timestamp at the same time.</span><br /><br /><span style="font-family: arial;">The Garmin 18LVC model speaks the NMEA protocol, which sends out every </span><span style="font-family: arial;">second the NMEA messages that have been previously selected with the PGRMO </span><span style="font-family: arial;">command. Only the GPRMC message is necessary to get the current date and </span><span style="font-family: arial;">time, and it is important to keep to amount of information sent out over the </span><span style="font-family: arial;">serial port every second to a minimum. Once the PGRMO command has selected </span><span style="font-family: arial;">the message types, the GPS will retain this configuration in its internal</span> <span style="font-family: arial;">memory.</span><br /><br /><span style="font-family: arial;">The output from the GPS can be seen using a terminal program, such as </span><span style="font-family: arial;">minicom, if properly configured at 4800 baud, no parity, 1 stop bit.</span><br /><span style="font-family: arial;"><br /><span style="font-weight: bold;">2. NTPD reference clocks</span></span><br /><br /><span style="font-family: arial;">The ntpd server supports several types of drivers. Here, the term 'driver'</span> <span style="font-family: arial;">has nothing to do with a 'kernel driver'. ntps drivers are low-level </span><span style="font-family: arial;">callback functions that are registered within the ntpd core and implement </span><span style="font-family: arial;">the access to several types of local clocks, such as GPSs , radio clocks,</span> <span style="font-family: arial;">etc. Each driver is identified by a pseudo-IP address identifier. The </span><span style="font-family: arial;">identifiers involved here are:</span><br /><br /><span style="font-family: arial;">127.127.20.x : NMEA Reference Clock driver</span><br /><span style="font-family: arial;">127.127.28.x : SHM (shared memory) driver</span><br /><br /><span style="font-family: arial; font-weight: bold;">2.1 NMEA reference clock</span><br /><br /><span style="font-family: arial;">The NMEA clock driver assumes that a GPS device sending out NMEA messages</span> <span style="font-family: arial;">is connected to the system via a serial port, named /dev/gpsX and its PPS </span><span style="font-family: arial;">signal, wired through the DCD pin, is accessible from a /dev/gpsppsX device.</span><br /><br /><span style="font-family: arial;">/dev/gpsX is actually a link to some /dev/ttySX serial device, and</span> <span style="font-family: arial;">/dev/gpsppsX is a link to a /dev/ppsX device which, in turn, is provided by </span><span style="font-family: arial;">the kernel PPS API. This API collects and distributes a precision kernel clock</span> <span style="font-family: arial;">information from/to userland programs, and supports some predefined client </span><span style="font-family: arial;">drivers, such as the DCD pin connected to a 8250 UART. The DCD pin is sensed </span><span style="font-family: arial;">using a new serial line discipline, named PPS, which is an extension of the</span> <span style="font-family: arial;">TTY line discipline. The sensing takes place during interrupt time, so it </span><span style="font-family: arial;">provides a very precise timestamping of the DCD events.</span><br /><br /><span style="font-family: arial;">This PPS API, also known as LinuxPPS, is not yet available in the kernel, so</span> <span style="font-family: arial;">a patch must be applied to the kernel mainline tree.</span><br /><br /><span style="font-family: arial;">Note the /dev/gpsppsX device is optional and if ntpd cannot open this device </span><span style="font-family: arial;">at startup, it will silently fall back to the NMEA-only functionality. This </span><span style="font-family: arial;">means that it will use the arrival time of the NMEA messages to discipline </span><span style="font-family: arial;">the system clock, which will result in a very poor precision, usually, worse </span><span style="font-family: arial;">than using a remote NTP server over the internet. Running ntpd in debug mode</span> <span style="font-family: arial;">will, however, log this condition.</span><br /><span style="font-family: arial;"><br />The /etc/ntp.conf must contain these lines:</span><br /><br /><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;">server 127.127.20.0 mode 1 minpoll 4 prefer</span><br /><span style="font-family: arial; font-weight: bold;">fudge 127.127.20.0 flag3 1 flag2 0 time1 0.0</span><br /></span><br /><span style="font-family: arial;">Here note that a single NTP device provides both the timestamp and the PPS</span> <span style="font-family: arial;">timing. The meaning of these flags are as follows:</span><br /><br /><span style="font-family: arial;">mode=1, means that only the GPRMC messages of the NMEA protocol will be</span> <span style="font-family: arial;">analyzed. flag3=1 tells ntpd to use the PPS line discipline of the kernel, </span><span style="font-family: arial;">and flag2=0 tells the driver to use the rising edge of the DCD signal to</span><span style="font-family: arial;"> signal the start of each second.</span><br /><br /><span style="font-family: arial;">In order to activate the PPS line discipline on the serial port connected to</span> <span style="font-family: arial;">the GPS, it is necessary to run the 'ldattach' utility, which actually will </span><span style="font-family: arial;">stay in the background to keep the serial port open and the discipline </span><span style="font-family: arial;">active:</span><br /><br /><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;"># ldattach pps /dev/ttyS0</span><br /></span><br /><br /><span style="font-family: arial; font-weight: bold;">2.2 SHM reference clock</span><br /><br /><span style="font-family: arial;">The SHM driver accepts delayed timing information from a System-V IPC shared</span> <span style="font-family: arial;">memory (with key "NTPx"). The timing information is written there by some </span><span style="font-family: arial;">external process, whatever it is. This process would read the timing </span><span style="font-family: arial;">information from the GPS and write it to the shared memory so that ntpd can </span><span style="font-family: arial;">process it. There are some user-space utilities that can do this job, for</span> <span style="font-family: arial;">example, gpsd and shmpps. Gpsd is a general-purpose daemon that has been </span><span style="font-family: arial;">designed to talk to most types of GPS models using a wide variety of </span><span style="font-family: arial;">protocols and, in addition, is capable of processing the PPS signals and </span><span style="font-family: arial;">sending timing information to ntpd via a shared memory device. Or, at least,</span><br /><span style="font-family: arial;">this is what it claims to do... because I couldn't achieve this.</span><br /><br /><span style="font-family: arial; font-weight: bold;">2.2.1 gpsd</span><br /><br /><span style="font-family: arial;">Actually gpds feeds two devices to ntpd, one with the absolute timestamp</span> <span style="font-family: arial;">parsed from the NMEA messages, or any other protocol supported by gpsd, and </span><span style="font-family: arial;">another feeding the PPS timing information. Ntpd sees both devices as two </span><span style="font-family: arial;">different SHM devices, so the ntp.conf file must be like this:<br /><br /></span><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;">server 127.127.28.0 minpoll 4</span><br /><span style="font-family: arial; font-weight: bold;">fudge 127.127.28.0 refid GPS</span><br /><span style="font-family: arial; font-weight: bold;">server 127.127.28.1 minpoll 4 prefer</span><br /><span style="font-family: arial; font-weight: bold;">fudge 127.127.28.1 refid PPS</span><br /></span><br /><span style="font-family: arial; font-weight: bold;">2.2.2. shmpps</span><br /><br /><span style="font-family: arial;">shmpps is a much simpler daemon that does exactly one job: detect the PPS</span> <span style="font-family: arial;">changes, get a timestamp for each change, and send them to ntpd via a single </span><span style="font-family: arial;">shared memory. No absolute time is passed to ntpd, so ntpd should still use </span><span style="font-family: arial;">a NMEA device (on the same serial port) to get the absolute time reference.</span><br /><br /><span style="font-family: arial;">So, /etc/ntp.conf must look like this:<br /><br /></span><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;">server 127.127.20.0 mode 1 minpoll 4 prefer</span><br /><span style="font-family: arial; font-weight: bold;">fudge 127.127.20.0 flag3 0 flag2 0 refid NMEA</span><br /><span style="font-family: arial; font-weight: bold;">server 127.127.28.0 minpoll 4</span><br /><span style="font-family: arial; font-weight: bold;">fudge 127.127.28.0 refid PPS</span><br /></span><br /><span style="font-family: arial;">Both gpsd and shmpps share a common problem: they detect DCD changes from</span> <span style="font-family: arial;">user-space, that is, a program that blocks on the TIOCMGET ioctl until a </span><span style="font-family: arial;">change is detected and then gets a timestamp. In this case, the latency is </span><span style="font-family: arial;">larger that in the PPS kernel API, where the timestamp is read at interrupt </span><span style="font-family: arial;">time.</span><br /><br /><span style="font-family: arial;">In conclusion, the LinuxPPS approach offers the best precision and</span> <span style="font-family: arial;">simplicity as it requires no external daemons, but the kernel must be </span><span style="font-family: arial;">patched. On the other side, the SHM devices offer worse precision but are </span><span style="font-family: arial;">easier to set up.</span><br /><br /><span style="font-family: arial; font-weight: bold;">3. How to enable support for PPS API</span><br /><br /><span style="font-family: arial;">Here I explain how to patch and build the software modules involved in</span> <span style="font-family: arial;">having GPS/PPS connected to ntpd. I also rebuilt the RPMs for these modules </span><span style="font-family: arial;">so that instalations is easier. The process involved in rebuilding the RPMs </span><span style="font-family: arial;">is also described below.</span><br /><br /><span style="font-family: arial; font-weight: bold;">3.1. Building the Linux kernel</span><br /><br /><span style="font-family: arial;">There are two ways to get LinuxPPS: via git or via patches.</span> <span style="font-family: arial;">The latest version of LinuxPPS is available only via the git repository </span><span style="font-family: arial;">and it contains an entire Linux kernel tree, but only for one</span><br /><span style="font-family: arial;">kernel version version (2.6.28-rc6 at the time of writing).</span> <span style="font-family: arial;">However, the patches are against a wider repertory of kernel versions, </span><span style="font-family: arial;">but the LinuxPPS implementations are older.</span><br /><br /><span style="font-family: arial;">I'd rather not use the kernel available from the git repository</span> <span style="font-family: arial;">because I prefer to stick to the same kernel version that is already </span><span style="font-family: arial;">installed in the target system, in my case, a Fedora 10 (2.6.27.5-117)</span> <span style="font-family: arial;">but, at the same time, I want to have the latest LinuxPPS implementation.</span><br /><br /><span style="font-family: arial;">So, I decided to patch the 2.6.27 kernel manually. This shouldn't take</span> <span style="font-family: arial;">much time, as the patches look quite straightforward to apply.</span><br /><br /><span style="font-family: arial;">First, I downloaded the entire git repository and diff'ed it against </span><span style="font-family: arial;">a stock 2.6.28-rc6 kernel, the resulting patch is the LinuxPPS implementation.</span><br /><br /><span style="font-family: arial;">To get the original kernel that comes with my Fedora 10, I had to </span><span style="font-family: arial;">download the source RPM for the installed kernel version (2.6.27.5-117) </span><span style="font-family: arial;">and install it. This must be done on the target PC.</span><br /><br /><span style="font-family: arial;">Then, I prepared a source tree for the patch and build:</span><br /><br /><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;"># cd ~/rpmbuild</span><br /><span style="font-family: arial; font-weight: bold;"># rpmbuild --target i686 -bp SPECS/kernel.spec</span><br /></span><br /><span style="font-family: arial;">Note the spec file already creates the kernel config file so there is no</span> <span style="font-family: arial;">need to configure the kernel to match our current settings.</span><br /><br /><span style="font-family: arial;">Just to test, I tried to apply the LinuxPPs patch directly to my 2.6.27.5</span> <span style="font-family: arial;">kernel but, of course, it didn't work as there were too many differences.<br /><br /></span><span style="font-family: arial;">Then, I split the patch into two parts: one containing the new files</span> <span style="font-family: arial;">specific to the pps implementation, and another for the existing kernel</span> <span style="font-family: arial;">files. The first sub-patch was applied successfully, and the second one </span><span style="font-family: arial;">had to be manually applied, but it wasn't too painful. At the end,</span><br /><span style="font-family: arial;">I diff'ed against the original 2.6.27.5 kernel and saved them to a patch file</span> <span style="font-family: arial;">(patch-2.6.27.5-ppsapi).</span><br /><br /><span style="font-family: arial;">Then, I configured the kernel to enable PPS support:</span><br /><br /><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;"> make menuconfig</span><br /><span style="font-family: arial; font-weight: bold;">Device drivers -> PPS support</span><br /><span style="font-family: arial; font-weight: bold;">* Enable high-resolution timestamps</span><br /><span style="font-family: arial; font-weight: bold;">* Enable 'ktimer' and 'line discipline' as modules</span><br /></span><br /><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;">General setup -> append to kernel version: "pps" to have this tag</span><br /><span style="font-family: arial; font-weight: bold;">on the kernel version string.</span><br /></span><br /><span style="font-family: arial;">Then I compiled the kernel and the modules</span><br /><br /><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;"># make</span><br /><span style="font-family: arial; font-weight: bold;"># make install</span><br /></span><br /><span style="font-family: arial;">Check that /etc/grub.conf has the correct entries (it should have)</span> <span style="font-family: arial;">so that when the system boots again, the new kernel is used</span><br /><br /><span style="font-family: arial;">Now, the kernel header files must be prepared for compilation of the</span> <span style="font-family: arial;">user-space tools:</span><br /><br /><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;"># cd /usr/include</span><br /><span style="font-family: arial; font-weight: bold;"># mv linux linux.orig</span><br /><span style="font-family: arial; font-weight: bold;"># mv asm asm.orig</span><br /><span style="font-family: arial; font-weight: bold;"># mv asm-generic asm-generic.orig</span><br /><span style="font-family: arial; font-weight: bold;"># ln -s /lib/modules/2.6.27.5pps/build/include/linux</span><br /><span style="font-family: arial; font-weight: bold;"># ln -s /lib/modules/2.6.27.5pps/build/include/asm</span><br /><span style="font-family: arial; font-weight: bold;"># ln -s /lib/modules/2.6.27.5pps/build/include/asm-generic</span><br /><span style="font-family: arial; font-weight: bold;"># cp /lib/modules/2.6.27.5pps/build/Documentation/pps/timepps.h .</span><br /></span><br /><span style="font-family: arial;">Note the timepps.h header is required for ntpd to detect the presence</span> <span style="font-family: arial;">of the PPSAPI in the Linux kernel. Otherwise, ntpd wouldn't complain </span><span style="font-family: arial;">and silently revert to using the NMEA protocol only.</span><br /><span style="font-family: arial;"><br /><span style="font-weight: bold;">3.2. Building the 'ntpd' server</span></span><br /><br /><span style="font-family: arial;">Now, it is time to patch and compile the NTP server (ntpd).</span> <span style="font-family: arial;">I used the latest available release, 4.2.4p7 at the time of writing.</span><br /><br /><span style="font-family: arial;">Initially, I used the nmea.patch from the LinuxPPS homepage</span> <span style="font-family: arial;">but the resulting ntpd failed to detect the PPS signal from the GPS. </span><span style="font-family: arial;">After some detailed debugging using gdb, I found out that the DCD </span><span style="font-family: arial;">interrupts from the UART were disabled by ntpd during initialization,</span> <span style="font-family: arial;">preventing the PPS signal from reaching the processor.</span><br /><br /><span style="font-family: arial;">A more detailed debugging showed that the call to tcsetattr() in the </span><span style="font-family: arial;">refclock_setup function, indirectly caused the DCD interrupts to be </span><span style="font-family: arial;">disabled, though the bit mask of c_iflag should not cause this problem.</span> <span style="font-family: arial;">I believe that changing some of the c_iflag bits causes the UART to</span><br /><span style="font-family: arial;">be incorrectly reprogrammed, perhaps this is why the ldattach utility</span> <span style="font-family: arial;">requires patching. So, I decided to patch ldattach so that the c_iflag </span><span style="font-family: arial;">settings are the same as the ones set by ntpd. This is a dirty hack, but </span><span style="font-family: arial;">it seems to be how the LinuxPPS patches work here.</span><br /><br /><span style="font-family: arial;">If you are experiencing problems and you suspect that PPS is not being</span> <span style="font-family: arial;">read by ntpd, try reading the IER register (Interrupt Enable Register) </span><span style="font-family: arial;">of your 8250 UART (or compatible) and check that bit 3 is set. The IER </span><span style="font-family: arial;">register is at offset 0x01 of the UART (i.e. address 0x3F9).</span><br /><br /><span style="font-family: arial;">To compile ntpd, download ntp-4.2.4p7 and apply the patch-ntp-4.2.4p7-ppsapi,</span> <span style="font-family: arial;">then configure it to enable the NMEA driver (ID 127.127.20.u):</span><br /><br /><span style="font-size:85%;"><span style="font-family: arial; font-weight: bold;"># ./configure --disable-all-clocks --disable-parse-clocks --enable-NMEA --enable-linuxcaps</span><br /></span><br /><span style="font-family: arial;">Additionally, to may want to enable support for SHM drivers, in case you </span><span style="font-family: arial;">want to experience with user-space drivers, which don't require kernel </span><span style="font-family: arial;">patching but are more likely to be affected by latencies and be less </span><span style="font-family: arial;">precise. If so, add the "--enable-SHM" argument to the configure command.</span><br /><br /><span style="font-family: arial;">Now, run 'make' and use the produced 'ntpd' driver and utilities.</span><br /><br /><span style="font-family: arial;">Remember that the Linux PPSAPI must be enabled in the kernel, and that the</span> <span style="font-family: arial;">correct kernel include files must be visible under /usr/include, as </span><span style="font-family: arial;">explained in the previous chapter.</span><br /><br /><span style="font-family: arial; font-weight: bold;">3.3. Building the ldattach utility</span><br /><br /><span style="font-family: arial;">The ldattach utility is used to set the line discipline associated to a</span> <span style="font-family: arial;">serial port. LinuxPPS uses a new line discipline, named PPS, that detects </span><span style="font-family: arial;">changes in the DCD line of the serial port and feeds those changes into </span><span style="font-family: arial;">the kernel PPS API.</span><br /><br /><span style="font-family: arial;">A small patch must be applied to ldattach, but it must be slightly</span> <span style="font-family: arial;">different to the one proposed in the LinuxPPS homepage. I believe the spirit </span><span style="font-family: arial;">behind this patch is to set the same terminal config both by ldattach </span><span style="font-family: arial;">and ntpd so that no change is done and the UART registers preserve the</span><br /><span style="font-family: arial;">DCD detection.</span><br /><br /><span style="font-family: arial;">ldattach is provided by the util-linux package in the Fedora 10</span> <span style="font-family: arial;">distribution.</span><br /><br /><span style="font-family: arial;">The easiest way is to install the source RPM, add the new patch in the</span> <span style="font-family: arial;">spec file and rebuild the rpm.</span><br /><br /><span style="font-family: arial; font-weight: bold;">3.4 Rebuilding the RPMs</span><br /><br /><span style="font-family: arial;">The affected RPMs are: kernel, kernel-headers, ntp, ntpdate, util-linux-ng</span> <span style="font-family: arial;">and their debug and devel variants.</span><br /><br /><span style="font-family: arial;">Rebuilding an RPM so that more patches are applied is quite straighforward.</span> <span style="font-family: arial;">Usually, an RPM that comes from a distribution package has already several </span><span style="font-family: arial;">patches that get applied when the rpm is built. All we have to do is add the </span><span style="font-family: arial;">corresponding pps patches to each spec file and rebuild the RPM.</span><br /><br /><span style="font-family: arial;">In general, the process is as follows: the source RPM is installed and this</span> <span style="font-family: arial;">installs a spec file in the SPECS directory, and a source tarball and the patch </span><span style="font-family: arial;">files under the SOURCES directory. The SPECS and SOURCES directory can be </span><span style="font-family: arial;">found under the buildroot directory of the rpmbuild utility, usually under </span><span style="font-family: arial;">~/rpmbuild. Then, the additional pps patch must be copied in the SOURCES</span><br /><span style="font-family: arial;">directory, and then edit the spec file to add the new patch file using a %patch</span> <span style="font-family: arial;">clause. Number the patch clause so that our patch is applied after the</span> <span style="font-family: arial;">others.</span><br /><br /><span style="font-family: arial;">Also, it is a good idea to add the 'pps' suffix in the release string of the</span> <span style="font-family: arial;">RPM, otherwise it would be impossible to tell a pps-capable rpm from a </span><span style="font-family: arial;">non-capable one.</span><br /><br /><span style="font-family: arial;">The kernel RPM is the most complex of these RPMs, and the spec file had to</span> <span style="font-family: arial;">be changed more:</span><br /><br /><span style="font-family: arial;">* The 'buildid' macro was defined to be ".pps"</span><br /><span style="font-family: arial;">* Patches are applied using the "ApplyPatch" macro</span><br /><span style="font-family: arial;">* The timepps.h header must be copied to <root>/usr/include when building</span><br /><span style="font-family: arial;"> the kernel-headers rpm.</span><br /><br /><span style="font-family: arial;">As described in the above paragraphs, before building the util-linux and ntp </span><span style="font-family: arial;">packages, the links to the new kernel include files must be done. </span><span style="font-family: arial;">Alternatively, the pps kernel rpm can be built and installed first, and then </span><span style="font-family: arial;">rebuild the rest of the packages.</span><br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com3tag:blogger.com,1999:blog-1135701757802805375.post-72743637626413819152009-06-15T17:09:00.004+02:002009-06-18T12:01:53.645+02:00Installing Fedora from a USB pendrive<span style="font-family:arial;">I had to install Fedora 10 on a PC without floppy disk and DVD/CD-ROM. The only choice is to boot from USB-HD which is supported by the BIOS. Here is the process, step by step:<br /><br />1. Install the livecd tools on a Linux PC:<br /><br /><span style="font-weight: bold;"># yum install livcecd-tools</span><br /><br />2. Download the DVD iso image of the Fedora distribution that must be installed on the target system. Pay attention to chooses the correct architecture (i686 or x86_64).<br /><br />3. Loop-mount the ISO image to a temporary directory:<br /><br /><span style="font-weight: bold;"># mkdir /mnt/tmp</span><br /><span style="font-weight: bold;"># mount -o loop <image.iso> /mnt/tmp</image.iso></span><br /><br />4. Insert the USB drive and make sure it is not mounted. Then , run the script that creates a live (bootable) CD:<br /><br /><span style="font-weight: bold;"># livecd-iso-to-disk --reset-mbr /mnt/tmp/images/boot.iso <image.iso> /dev/sdb1</image.iso></span><br /><br />5. Mount the USB drive and copy the installer and the ISO image to the USB drive:<br /><br /><span style="font-weight: bold;"># mkdir /media/<usbdisk><usb>usbdisk/images</usb></usbdisk></span><br /><span style="font-weight: bold;"># cp /mnt/tmp/images/install.img /media/<usb>usbdisk/images</usb></span><br /><span style="font-weight: bold;"># cp <image.iso> Fedora-10-i386.iso /media/<usbdisk>usbdisk/</usbdisk></image.iso></span><br /><br />6. Unmount the USB drive, and insert it on the target system. The Anaconda installer must show up shortly after power-up. Select Install Media from "Hard drive" and proceed as in a normal DVD installation.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-24807792711119812452009-06-15T16:56:00.002+02:002009-06-15T17:08:04.661+02:00More problems with OWAMP behind NAT<span style="font-family: arial;">Well, not all NAT problems were solved with the fixes detailed in the previous post. Indeed, the server does not check that the OWAMP client is sending from the address specified in the "receiver IP address" field of the Request-session message, but there are some outstanding problems.<br /><br />First, the server will create a new socket for the test session, and will attempt to bind that socket to the address specified for the sender or the receiver in the request-session message. This address is, of course, the public address of the server as the message was created and sent by the client. Obviously, the server will fail in binding that socket to a public address it is not in none of its interfaces.<br /><br />Second, the client must specify its actual public address in the sender/receiver IP address field of the request-session message, otherwise, the other end will not be able to send test packets to that address.<br /><br />Therefore, some modification is required in the OWAMP code. On the client side, the public address must be specified in the messages. Also, on the server side, the server should bind the test socket to the private address, regardless of the address specified in the session-request message.<br /><br />The client needs to know its actual public address on the public side of the NAT, which is specified by the new '-x' option of the owping program. The owampd daemon only needs to know if it is running behind a NAT or not, but it doesn't know the public address, makeing the server setup more simple. The -N flag is used in this case.<br /><br />So, the extra requirement is for the client to know its public address, which can be obtained by a STUN client or a similar service.<br /><br />Now, it definitely works fine, with both the server and the client running behind their respective NATs.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com3tag:blogger.com,1999:blog-1135701757802805375.post-53117562893167897192009-06-05T14:08:00.002+02:002009-06-05T14:21:11.766+02:00OWAMP problems behind NAT<span style="font-family: arial;">I've been doing some tests with the OWAMP protocol (One-way Active Measurement Protocol) and it turns out that it is not possible to ow-ping a server when the owamp client (owping) is behind a NAT. The client returns a 'server denied access' error.<br /><br />OWAMP is one of those non-NAT-traversable protocols, such as SIP, as it passes endpoint IP addresses in the protocol messages. If the client is sitting behind a NAT, the source address passed is not the same as the actual source IP address as seen by the server. During a test session request stage, the owamp server checks that both addresses are the same, in order to prevent attacks, si one would think that the OWAMP protocol is unusable if the client is behind a NAT.<br /><br />But not. It happens that the owamp server only checks the addresses in open mode. So, if we enable the authenticated mode, for example, the check is omitted and everything works.<br /><br />To work in authenticated mode, all you need to do is to setup a common passphrase in both sides so that the client gets authenticated. The passphrases are kept in the owampd.pfs file and are generated by the 'pfstore' utility:<br /><br /><span style="font-weight: bold;"># pfstore -f /usr/local/etc/owampd.pfs testuser</span><br /><br />Then, run owampd so that it loads the pfs file (using the -c option).<br />Repeat the same pfstore action on the client machine, and then ping the server:<br /><br /><span style="font-weight: bold;"># owping -A A -u testuser -k /usr/local/etc/owampd.pfs <owamp-server></span><br /><br />and this works.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-48009153325677670782009-05-25T13:02:00.002+02:002009-05-25T13:06:43.269+02:00Time loop in VMware guest<span style="font-family: arial;">If the guest VM should always boot with the same date/time on the RTC, two actions must be done:<br /><br />1. Edit the .vmx file of the guest VM, and add a line like this:<br /><br /><span style="font-weight: bold;">rtc.starttime = "1238765436"</span><br /><br />where the number is a 1970-based epoch, in seconds.<br /><br />2. Boot the guest VM, and run VMtools. Disable the host-to-guest time synchronization, and reboot the guest VM.<br /><br />Then, the guest VM will always boot with the same date and time in the calendar.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-67681698940922073192009-05-20T10:36:00.004+02:002009-05-20T10:47:41.073+02:00Setup a Samba server<span style="font-family:arial;"><br />Edit /etc/samba/smb.conf and add set the basic settings in the 'global' section. The only important item is the "security=user" option.<br /><br />The important thing is to enable the samba users with:<br /><br /><span style="font-weight: bold;"># smbpasswd -a <username></username></span><br /><br />which adds (-a) a new user to the samba user database, and sets a new password to it.<br />Not doing this causes a 'permission denied' error in the client when attempting to mount it.<br /><br />The syntax to mount a samba share is:<br /><br /><span style="font-weight: bold;"># mount -t cifs -o user=<username>,pass=<password> //server/resource <mountpoint></mountpoint></password></username></span><br /><br />In the /etc/samba/smbusers file, it is possible to map sambe user names to UNIX user names, but it is not required to do so. By default, samba does a one-to-one mapping.<br /><br />If we want to announce the samba shares on the network, it is necessary to start the 'nmb' service. Use the normal chkconfig interface to start smb and nmb on startup:<br /><br /><span style="font-weight: bold;"># chkconfig --add nmb</span><br /></span><span style="font-weight: bold;font-family:arial;" ># chkconfig --level 3 nmb on<br /></span><span style="font-weight: bold;font-family:arial;" ># chkconfig --level 4 nmb on<br /></span><span style="font-weight: bold;font-family:arial;" ># chkconfig --level 5 nmb on<br /></span><br /><br /><br /><span style="font-family:arial;"><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-68541473578351278582009-05-12T19:34:00.002+02:002009-05-12T19:56:12.035+02:00Set up VMware server on a x86_64 system<span style="font-family: arial;">Setting up a 1.0.8 VMware server on a 64-bit PC creates some problems because VMware is distributed in binary form only (rpm packages) for 32-bit i386 architectures .<br /><br />In my case, the target distribution is a Fedora 10 x86_64, so the first step was installing the 32-bit base libraries, that is, glibc, X11, etc. This can be easily done with the 'yum' installer:<br /><br /><span style="font-weight: bold;"># yum install glibc.i686 libgcc.i386 lidstdc++.i386 zlib.i386</span><br /><span style="font-weight: bold;"># yum install libX11.i386 libXtst.i386 libXrender.i386 libXt.i386</span><br /><br />Also, some Perl modules are required for the VMware installer to work properly:<br /><br /><span style="font-weight: bold;"># yum install perl-ExtUtils-Embed</span><br /><br />which is not 32-bit specific, of course.<br /><br />Now, we are in a position where the VMware-server package may be installed:<br /><br /># rpm -ivh VMware-server-1.0.8.i386.rpm<br /><br />If we hadn't installed the i386 libraries, the vmware-server rpm could be installed anyway but problems would appear when running the vmware-config.pl script, which runs the vmware-ping utility, which is a 32-bit one.<br /><br />Now, some patches to the vmware distribution must be installed. In my case, the vmware-any-any-117d could not be installed, as it returned an error when compiling the kernel modules (something about an already defined symbol). The update that worked fine was 'vmware-update-2.6.27-5.5-2.tar.gz' but this depends on the particular kernel version that runs in my target system.<br /><br />Installing this update is quite straightforward, however, this tarball contains a compiled 'update' executable which is intended for 32-bit installations. I decided to delete it and recompile manually before running the update:<br /><br /><span style="font-weight: bold;"># tar xfz vmware-update-2.6.27-5.5-2.tar gz</span><br /><span style="font-weight: bold;"># cd vmware-update-2.6.27-5.5-2</span><br /><span style="font-weight: bold;"># rm update</span><br /><span style="font-weight: bold;"># gcc update.c -o update</span><br /><span style="font-weight: bold;"># ./runme-pl</span><br /><br />The 'runme.pl' script automatically launches the 'vmware-config.pl' script. I accepted all the proposed default answers, except for the NAT networking, which I answered 'no', the host-only networking -also 'no'- and the TCP port to listen to, which is by default 904, but I changed it to 902 to match the default port the clients connect to (why this discrepancy in the defaukt values?).<br /><br />If I had accepted the 904 port in the server side, then the client wouldn't connect to the server and would return a 'connection refused' error. Fortunately, the client connectiomn dialog accepts an optional port number after the server IP, i.e. myserver:904.<br /><br />After this, the client still returned an error when attempting to connect to the client: 'login incorrect (username/password)'. After some investigations, I found out that the problem was in the PAM libraries. Again, only the 64-bit pam libraries were installed, but vmware expects the 32-bit libraries, so the user authentication failed. Unfortunately, the pam i386 and x86_64 libraries cannot coexist simultaneously, because there are some common files provided by both packages (i.e. the man pages). So, the workaround is to download the pam package and its dependencies, and do a force install:<br /><br /><span style="font-weight: bold;"># yumdownloader --resolve pam.i386</span><br /><span style="font-weight: bold;"># rpm -ivh --force *.rpm</span><br /><br />Also, we must configure the proper path to the pam libraries.<br />Edit <span style="font-weight: bold;">/etc/pam.d/vmware-authd</span> :<br /><br /><span style="font-weight: bold;">#%PAM-1.0</span><br /><span style="font-weight: bold;">auth sufficient /lib/security/pam_unix2.so shadow nullok</span><br /><span style="font-weight: bold;">auth required /lib/security/pam_unix_auth.so shadow nullok</span><br /><span style="font-weight: bold;">account sufficient /lib/security/pam_unix2.so</span><br /><span style="font-weight: bold;">account required /lib/security/pam_unix_acct.so</span><br /><br />After all this, vmware server runs correctly and clients may connect.<br /><br />To install the vmware client (a.ka. vmware-server-console), all I had to do is install the client rpm:<br /><br /><span style="font-weight: bold;"># rpm -ivh VMware-server-console-1.0.8-126538.i386.rpm</span><br /><span style="font-weight: bold;"># vmware-config-server-console.pl</span><br /><br />and that's all.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-73735049680920727032008-12-16T14:29:00.002+01:002008-12-16T14:32:35.312+01:00Using code sourcery to build a toolchain<span style="font-family: arial;">Though it looks quite primising, it was very tough trying to build a cross toolchain using code-sourcery.<br /><br />They provide prebuilt binary toolchains, but I none with the gcj compiler, so I had to use their build scripts to build my own toolchain.<br /><br />The script they provide wasn't designed to run on any machine. In fact, it does not run on a machine different to the author's one. There are many hardwired directories and custom build is very tough. Though I tried to understand it, at the end some unexplainable errors appeared.<br /><br />In conclusion, code sourcery for custom toolchains is not an option nowadays.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-75342556968079899252008-12-16T14:20:00.002+01:002008-12-16T14:28:37.230+01:00Testing the MIKA JVM<span style="font-family: arial;">Downloading mika requires subversion as there seem to be no tarballs available for download.<br />Subversion requires port 3690, plus other tools.<br /><br />Mika's repository is :<br />http://www.k-embedded-java.com/mika/trac/wiki/DownloadMikaNew<br /><br />The dependencies are: jikes and jam.<br /><br />Jikes is a java compiler from IBM. It is available as an rpm for FC4. No problems installing it.<br /><br />Jam is a set of Ant scripts for building java applications. Just run configure, make, and make install. However, my FC4 has some broken links: /etc/alternatives/java_sdk_exports.<br />I had to edit the link to point to the installed JDK: /usr/lib/jvm/java-1.4.2-gcj<br /><br />To build mika, I had to edit the <span style="font-weight: bold;">Configuration/cpu/arm</span> file, where the toolchain executables are defined. It wrongly assumes that the compiler was arm-linux-gcc, ignoring the CC variable.<br /><br />To run the build process, type:<br /><span style="font-size:85%;"><span style="font-weight: bold;">ant -DPLATFORM=default -DJAM.PLATFORM=arm-linux</span><br /> </span><br /> Results are similar to the Skelmir's VM but with a different balance: setVisible is 3 times faster, but the screen creation is 3 times slower. The net performance is more or less the same.</span><br /><br /><span style="font-family: arial;">Mika looks quite lightweight in terms of the flash footprint required.</span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-67616577137863731652008-12-16T12:52:00.002+01:002008-12-16T14:19:56.566+01:00Using crosstool-ng<span style="font-family: arial;">The objective is to build a valid toolchain for ARM that supports gcj with the GTK peer classes.<br />The latest gcc is required (version 4.3) which has the latest GNU classpath version, and still evolving.<br /><br />I tried several options to build a toolchain: crosstool, code-sourcery, etc, but crosstool-ng is the only one that was able to build a 4.3 toolchain without problems. The original crosstool was able to build a toolchain up to gcc 4.1.1/glibc 2.3.2. Newer versions of gcc or glibc did not compile.<br /><br />Building and installing crosstool is very straightforward. Create a temporary directory, untar the tarball (crosstool-1.3.0) and run:<br /><br /><span style="font-weight: bold;font-size:85%;" >mkdir /usr/crosstool-ng (as root)<br />chown <regular-username> /usr/crosstool-ng<br /><br />./configure --prefix=/usr/crosstool-ng<br />make<br />make install<br /></span><br />and then add /usr/crosstool-ng/bin to the PATH variable.<br /><br />The toolchain to be built is configured by running:<br /><span style="font-weight: bold;font-size:85%;" >ct-ng menuconfig<br /></span><br />There are some important hints:<br /><br />1. The build tuple must be:<br /><span style="font-size:85%;"><span style="font-weight: bold;">CT_BUILD=i686-pc-linux-gnu</span><br /> </span>which does not mean that the links i686-pc-linux-gcc -> gcc must be created. Crosstool-ng already creates its own wrapper scripts for this.<br /><br />2. EABI<br />Defines how code is generated according to some specific conventions. gcc 4.3 is able to generate EABI-compliant code. All the user-space code must be compiled with the same EABI version, and the kernel must be compiled with EABI support if the user-space code is also EABI.<br />A toolchain may be created with EABI support, but this does not necessarily mean that the code generated is EABI. Some gcc flags control this. The glibc library was compiled using EABI, therefore the kernel must be compiled with EABI support, otherwise the 'init' program would not be loaded by the kernel.<br /><br />3. Kernel headers<br />If we are using a new kernel (i.e. 2.6.1 or greater) then the best option is to use the sanitized kernel headers provided by the kernel's makefile. Crosstool-ng automatically invokes this makefile to generate the sanitized header set. By the way, using the latest 4.3 gcc, requires a late glibc (2.6 or 2.7), which in turn, requires a 2.6 kernel.<br /><br />4. glibc ports<br />If the option "use glibc ports" is not enabled, the glibc compilation may return an error like "target arm is not supported". As of glibc 2.5, the architecture-dependent stubs of glibc are distributed as separate tarballs.<br /><br />5. Dependencies<br />gcc 4.3 requires mrfp library version 2.3.2 or greater, which requires automake > 1.10 and autoconf > 2.60. There are no rpms for these versions for FC4, so I had to download the sources and install them manually<br /><br />6. Enabling gcj/java support<br />Must enable the java language.<br />Also, the specific java/gtk support must be enabled in the gcc build flags:<br /><span style="font-size:85%;"><span style="font-weight: bold;"><br />CT_CC_EXTRA_CONFIG=</span><br /> <span style="font-weight: bold;">--enable-java-awt=gtk --enable-gtk-cairo --disable-gtktest </span><br /> <span style="font-weight: bold;">--x-libraries=<path-to-x>/lib --x-includes=<path-to-x>/include/X11</span><br /> </span><br />note that the GTK peer classes provided by GNU classpath require some X libraries, though the GTK compiled uses the Directfb frontend, not the X-lib frontend. It seems that it still uses some X libraries for rendering functionality or something like that. Hopefully, these libraries are not required in the target system. In particular, it requires the libXtst, libart, libXrender and libXrandr libraries.<br /><br />It is important to set the pkg-config variables, so that the configure scripts of gcc and its subprojects are able to find the GTK libraries:<br /><br /><span style="font-size:85%;"><span style="font-weight: bold;">export PKG_CONFIG_PATH=<path-to-gtk>/usr/lib/pkgconfig</span><br /> <span style="font-weight: bold;">export PKG_CONFIG_LIBDIR=<path-to-gtk>/usr/lib/pkgconfig</span><br /> </span><br />the pkgconfig files (.pc) must be placed in these directories. Note that the 'prefix' keys in these files must be set to the location of the installed GTK files <path-to-gtk->/usr.<br /><br />there are GTK include files in different locations, such as: usr/lib/glib-2.0/include.<br /><br />7. Restarting crosstool-ng<br />Crosstool has a nice feature that allows the process to be restarted at some specific stage, thus speeding up the trial and error build process. However, this functionality must be enabled in the menuconfig, and this functionality has some limitations:<br />1. changing some configuration items (i.e. gcc build flags) take no effect if the process is restarted. The entirre build process must be started from the beginning.<br />2. Do not break crosstool-ng while downloading the tarballs. If so, clean the tarballs directory as some files may be corrupted.<br /><br />8. X libraries<br />Some GTK peer classes cannot be compiled because they require a GTK library compiled with xlib support. For instance, the cairo-xlib.h and gdk/gdkx.h include files are required.<br /><br />After all these steps were solved, a toolchain with gcj/gtk support was built without errors.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com2tag:blogger.com,1999:blog-1135701757802805375.post-42766146040022674512008-11-06T09:28:00.003+01:002008-11-06T09:44:36.487+01:00SIGIO terminates program<span style="font-family: arial;">I have compiled the IPC/nt library (inter-process communications) for the i686 architecture, on a CentOS 5.2 release. Compilation was ok, but at runtime, the programs that link with this library die unexpectedly when receiving data from the IPC layer. The message shown is "I/O possible".<br /><br />Of course, this did not happen on the original embedded systems this library was originally developed for, so I assume something has changed. There are two possibilities:<br /><br />1. getpid() no longer returns the pid of the calling thread, but the pid of the entire process.<br />2. the default action of the SIGIO signal has changed, causing the program to terminate rather than ignoring the signal.<br /><br />The mechanism that IPC/nt uses to work with asynchronous sockets is the following: a thread listening to SIGIO signals is created. Any asynchronous socket opened sets the owner of the SIGIO signals to the thread's pid (hence the getpid issue here). When data arrives (I/O is possible), a signal is sent to the thread which wakes up the other threads listening on the sockets using a mutex/condition variable.<br /><br />Then I came across this note that explains what is happening (though it does not explain exactly what has changed):<br /></span> <pre style="font-family: arial;"><span style="font-size:85%;">If a nonzero value is given to F_SETSIG in a multi-threaded process running with <br />a threading library that supports thread groups (e.g., NPTL), then a positive value<br />given to F_SETOWN has a different meaning: instead of being a process ID identifying<br />a whole process, it is a thread ID identifying a specific thread within a process. <br />Consequently, it may be necessary to pass F_SETOWN the result of gettid<a href="http://manpages.ubuntu.com/manpages/hardy/man2/gettid.html"></a> instead<br />of getpid(2) to get sensible results when F_SETSIG is used. (In current Linux threading<br />implementations, a main thread’s thread ID is the same as its process ID. This means that <br />a single-threaded program can equally use gettid(2) or getpid(2) in this scenario.)<br /></span><br />So, I replaced the call to getpid() by a call to gettid(), which I have proven it is<br />also backwards compatible. It works fine.<br /><br /></pre>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-39050429072567397822008-07-22T15:53:00.002+02:002008-07-22T16:02:48.566+02:00Testing qemu<span style="font-family: arial;">The goal is to run ARM software on a Linux PC.<br />I download the latest qemu, which is 0.9.1 today.<br /><br />There isn't much fun here, just the usual stages:<br /><br /><span style="font-size:85%;"><span style="font-weight: bold;">./configure --target-list=arm-linux-user</span><br /> <span style="font-weight: bold;">make</span><br /> <span style="font-weight: bold;">make install (as root)</span><br /> </span><br />Qemu requires the run-time libraries to run. The quickest way is to explode an entire package release to a root directory:<br /><br /><span style="font-size:85%;"><span style="font-weight: bold;"># tpkg-explode main tempdir</span><br /> </span><br />Warning: the postinstall scripts of the packages are not executed, which is not very important for most of the packages but the library cache file. So I manually edit etc/ld.so.conf and add the library paths there /usr/TCSL/lib.<br /><br />Then I rebuild the library cache:<br /><span style="font-size:85%;"><span style="font-weight: bold;"># ldconfig -r /<path-to>/tempdir</span><br /> </span><br />and now I can run any ARM program:<br /><br /><span style="font-size:85%;"><span style="font-weight: bold;"># qemu-arm -L tempdir tempdir/usr/TCSL/bin/unsls</span><br /> </span><br />It works great, but it does not detect memory overwrites or leaks (who told me that it could?).<br />It's fine, but not as a memory checker for ARM, as I intended.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com0tag:blogger.com,1999:blog-1135701757802805375.post-57793533125767068152008-07-22T15:02:00.000+02:002008-07-22T15:37:24.293+02:00Cross-compiling Valgrind<span style="font-family:arial;">Valgrind is available for other architectures other than the i386/PC. Unfortunately it is not ported to the ARM processor family, but it still can be useful for the PPC targets I use.<br /><br />First, I download the latest version of valgrind (today is 3.3.1).<br /><br />Before configuring, a word of advice: the prefix option does not work as usual. The absolute prefix path gets hardwired in the valgrind code so that valgrind is able to find other tools such as memcheck, etc. So, the prefix path must be used with the target distribution in mind. As I plan to install valgrind on a network directory and mount it via NFS from the target unit, I will set 'prefix' to<br />the name of the mountpoint.<br /><br />The reason for not installing valgrind on the target is because the size of the file is quite big (more than 50MB) as valgrind requires that his own binary files are not stripped. This is explained in the README_PACKAGERS file.<br /><br />Apart from setting the usual CC variable:<br /><span style="font-size:85%;"><span style="font-weight: bold;">/configure --host=ppc-linux --prefix=/ppc/tools/valgrind --disable-tls</span><br /> <span style="font-weight: bold;">make</span><br /> <span style="font-weight: bold;">make install</span><br /> </span><br />TLS is not provided by my kernel version, so it is disabled here.<br /><br /></span><span style="font-family:arial;">At this point, valgrind runs but it reports an error that it is unable to do some binary code substitutions (i.e. the strlen function). It suggests that the linker may be stripped /lib/ld-·so, which is true.<br /></span><br /><span style="font-family:arial;">Having the unstripped version of 'ld' requires compiling the entire glibc RPM. For my platform, the glibc is installed as an RPM. Then, I download and install the source RPM.<br /><br /></span><span style="font-family:arial;">The binary files get automatically stripped by rpmbuild at the end of the <span style="font-weight: bold;">%install</span> stage. The <span style="font-weight: bold;">"__os_install_post"</span> macro contains the script that must be run at that point. This scriptlet calls the <span style="font-weight: bold;">brp-strip-shared</span> script, which lives under <span style="font-weight: bold;">/opt/eldk/usr/lib/rpm,</span> and strips all the shared libraries found in the installation directory. Unfortunately, my installation of ppc-rpm does not allow this scriptlet to be edited, so the only option is to add a condition at the top of the </span><span style="font-family:arial;"><span style="font-weight: bold;">brp-strip-shared </span>scripts so that if the DONT_STRIP_SHARED variable is set, then exit the script immediately. The spec file must be edited so that this variable is defined.</span><br /> <br /><span style="font-family:arial;">Then, I modify the <span style="font-weight: bold;">glibc.spec</span> file where, at the end of the <span style="font-weight: bold;">%install</span> section, there is a piece of code that strips all shared libraries but the 'libpthread' library. I just add the 'ld' library as an exception to the strip process, and then I rebuild the RPM.<br /></span><span style="font-family:arial;"><br /><span style="font-size:85%;"><span style="font-weight: bold;">export PATH=/opt/eldk/usr/ppc-linux/bin/:$PATH</span><br /> <span style="font-weight: bold;">ppc-rpmbuild -ba glibc.spec</span><br /> </span><br />Do not forget to increase or change the version of the RPM so that it is installed in the target successfully.<br /><br />A simpler workaround is to copy the <span style="font-weight: bold;">ld-x.y.z.so</span> file manually, in this case, use 'cp -f' as the file is always in use.<br /><br /></span>pela-suroshttp://www.blogger.com/profile/07522894660925331288noreply@blogger.com4