ImaginationOverflow

The knowings and experiences of a group of developers.

Monthly Archives: October 2011

Where the hell is XML successor?

Today me and some colleges were discussing random stuff as usual, somewhere in the middle someone said something about XML, I  yawn (as always) and a discussion started about my XML hate relationship.

When I learnt XML (2 or 3 years ago) I loved it, it was easy to understand, very declarative and readable. But back in those days I knew nothing about the computer world, I only knew java and C, I didn’t knew what the hell was design patterns, code conventions, pretty code, etc.

When I learnt JSON I didn’t really understand its purpose so it was one more way to send data from one place to another.

Today I’ve a different view on how code and information should look like: namely, simple, concise, easy to read and the most important should be easy to write (low on chars). This last characteristic is probably the most important since its less memory used to save the data or less bandwidth used to exchange data, etc.

At first JSON seems perfect, it’s much more readable than XML and more compact. Then there is the underground MGraph by Microsoft with a really appealing syntax and is more compact than JSON, the problem is its inventor, Microsoft. The hardcore open-source community would never accept using something made by Microsoft so an overall usage would be automatically denied. Finally there is the YAML, my shame in all of this, since until today I never tried to found out what it was.

Let’s see it first, YAML version:

Course:
  Name: Web Programming
  Teacher: ATeacher
  Year: 2011
  Grades:
  - Student: Student A
    Value: 17
  - Student: Student B
    Value: 18

The XML version;

<Course Name="Web Programming" Year="2011">
  <Teacher> ATeacher </Teacher>
  <Grades>
     <Grade Value = "17"> Student A</Grade>
     <Grade Value = "18"> Student B</Grade>
  </Grades>
</Course>

Is it really necessary more comments? No, I fell in love with this new way to show data. Maybe I’m going to make a blog series on this, seems a really nice thing.

YAML have all the characteristics that I enumerated before, people will say that XML will always be here, because things like HTML, config files and schema support. The question is, isn’t XML a bit outdated? Its syntax didn’t get better or evolve with the technology, it’s pretty much the same thing that SGML (invented in the 60s), are we too accommodated with XML? Since when did the computer world stop searching for the best next thing? Think on it.

Resources:

YAML official site
MGraph the next xml

AVR32 First Contact, First Code

So this is the third of the AVR32 4 dummies post series, you can check the other blog posts on:

I finally started coding some stuff, namely the board startup, the context switch, the interrupt controller driver and the system timer driver. Along the way I discovered more things about the architecture, namely where the AVR32 call convention is documented. The Atmel support, when I contacted them to know about the call convention, forwarded me to the IAR compiler Reference Guide. I loled and checked if my reverse engineering discovers were right (they were).

Before any more comments and discoveries, here it is the overall description on my development board and CPU:

Board: EVK1104:

  • CPU : AT32UC3A3256, AVR32.
    • RAM: 128Kb.
    • ROM: 256Kb.
    • SPI: 6.
    • UART: 4.
    • I2C: 2.
    • ADC: 10bits (max resolution), with 8 channels, up to 384 ksps.
    • Speed: up to 66MHz.
  • 16 bit DAC.
  • SD card.
  • 256 Mbyte external RAM.
  • 256 Mbyte external ROM.
  • Touch Wheel for QMatrix.
  • 240×320 RBG LCD

Both the board and CPU websites are linked in the previous list, for more information about the EVK1104, download the Schematics and BOM since there is no user manual. The BOM (Bill of materials) have all the hardware used to create a dev board, so it’s the first place where you should go to find out what kind of hardware is “under the hood” or if you want a specific datasheet.

AVR32 Bootloader

A bootloader is a simple program or piece of code that initializes the hardware so that an operating system or application can run. The bootloader can be “called” by the reset event handler or, as usual, be part of the reset event handler. On AVR32 architecture the reset event is the only event that is statically mapped on the memory layout, at the address 0x80000000.

The bootloader present in the Atmel software libraries for my dev board follows the same principles on the bootloaders that I’ve done before for the ARM architecture:

  1. Set stack pointer.
  2. Set other system required configurations.
  3. Load the .data section (initialized variables).
  4. Set the .bss section to 0(uninitialized variables).
  5. Load .text to RAM (optional, used for better performance)
  6. Run the the global instances constructors (c++ only).
  7. Call main (or a operating system boot)

The loads are simple memcpy, from one place to another, in most of the cases from Flash to RAM.

I didn’t do my own bootloader for the board, I used one of the Atmel startups and added what I needed, namely the constructors call (MOS is in C++). I did the same for the linker script, I grabbed the Atmel ldscript and corrected some errors on it, again because of my C++ code (on the default linker scripts, although the C++ init arrays are present, the global variables are misplaced and without any change the init and fini addresses are always the same, for more information about C++ initialization check out the previous link of IAR compiler reference).

You can check out the startup and the ldscript via github.

Context Switch

The context switch is the operating system routine responsible for changing threads, this is done by saving the context of the current thread and loading the context of the next thread. A thread’s context is the minimal information needed to represent a thread’s execution state at the time of the switch. The context should be sufficient so that when a thread is switched its previous execution state can be restored and the thread execution continues like nothing happened.
In CPUs without MMU, FPU, etc, the thread context is reduced to the CPU registers, in AVR32 from R0-R15.

MOS has two ways to switch threads:

  • By interrupt: when an interrupt occurs the system checks if it is time to switch threads (MOS have a time-slice scheduler), if so the system changes the current thread context pointer (this context is stored on the epilogue of the interrupt and passed to system interrupt service routine) to the next thread context. With this way the context switch only happens when the interrupt returns.
  • By thread choice: when a thread doesn’t have nothing to do or is waiting to be signalized it calls kernel primitives like Yield, Sleep or Wait (in synchronizers), this functions can trigger a context switch.

Because of this feature, the two context switch implementations have to be linked to each other. Since the AVR32 interrupt handling automatically stores the r8-r12 registers, the “normal” context switch must save this registers first by the same order that is done on the interrupt. After that it’s needed to save the rest of the registers r0-r7, lr and pc. You’re probably asking why the sp (stack pointer) isn’t saved, the stack pointer is the only pointer that MOS has to know where the thread context is, so the stack pointer is saved on the thread instance, so that further switches can restore the thread context.

I’ve already pushed the AVR32 thread context and the context switch to github, go check it out.

If you check the context switch code, you will see that I don’t store all registers. Actually what I’m doing in some cases is to move stack pointer further down. For instance when the “normal” context switch is called, the caller must preserve all the scratch registers and since the context switch doesn’t use any local variable, I don’t need to store them (they are already stored in the thread stack by the caller of the context switch) so I move the stack  pointer down, saving precious memory accesses.

But when I restore the next thread I restore all the registers, the main reason is because it’s “impossible” to know how the next thread switched (by interrupt or by the normal way) so I simply restore them all.

On another note, the status register is also saved on the context switch, again this isn’t needed when thread is switched using the “normal” way, but when switched via interrupt is mandatory.

Final Comments

When doing the context switch I felt a bit forced to do things the AVR32 way, since the interrupt handling system stores automatically R8-R12 registers into the system stack. So MOS port will not take advantage of the Application mode, and all the code is going to run on System mode. After browsing the Atmel ldscript I felt a little disappointing on the development leftovers that were present and the fact that they don’t even bother to test the CPU that I’m using with C++, I can say this because the global constructors pointers are not correctly configured on the ldscript.
Also, In a previous post I hysterically talked about the status register being mapped on memory, I was wrong, you need to use special assembly instructions to read and write things to the SR.

On the next post I’ll talk about the interrupt handling system, the UC3 peripherals and the device drivers construction.

Feel free to ask any question or comment, see you next time.

Micro Operating System Overview, Port to AVR32

In this post I’m going to talk about the operating system that I made with the help of Sorcha. This post is the second on the AVR32 for dummies post series.

So the Micro Operating System or MOS is a simple, source readable operating system, or I believe so xD.

MOS is in C++ and divided in three main layers, System, Kernel and HAL (Hardware Abstraction Layer):

The System layer is the operating system development API, just like any operating system this layer is “responsible” for the application portability, by having all the possible software and hardware abstraction. This description of the System layer is my future goal, today this layer has little code base and little device drivers support.

The Kernel layer is where all my operating system knowledge is into. The kernel is based in two infra-structures that I studied before, eCos and user-threads.
The Kernel layer is divided in three sub-layers, InterruptController, Threading and Concurrent.

  • The Concurrent sub-layer is where all the synchronization primitives are implemented, things like atomic operations and synchronizers.
  • The Threading sub-layer is responsible for the Threads handling, it contains only two classes UThread and UScheduler.
  • The InterruptController is where the system delegate interrupt handling and treatment.

The HAL layer is the only one that knows what kind of hardware the system is running on. This layer is yet divided by two sub-layers, Target and Architecture Abstraction.

  • The Target Abstraction, is the only software component that knows on witch specific hardware the system is running on, so this sub-layer is responsible for all the hardware configuration and management.
  • The Architecture Abstraction is the sub-layer that knows on what kind of architecture the system is running. This sub-layer implements all system operations that it can, things like enable and disable interrupts, boot loaders, interrupt traps and the context switch.

Today MOS only supports ARM7 architecture. There is some LPC2xxx device drivers on it, and only ran on a development board the yaab2294 (LPC2294 with 8mb ram/rom expansion).

But that is about to change, with this port to AVR32 the operating system will be in two different architectures, processors and boards \o/.

The only thing I need to do is port the HAL layer to the AVR32 architecture and to my board (EVK1104, UC3 processor). In the next posts I’m going to talk more specifically on what is needed to do to port mos to another architecture and board.

MOS doesn’t have a pretty programming interface yet, but where’s some of it’s features:

  • Clean code, based on managed languages conventions (mainly C#) so that managed programmers don’t run away when they see the first code lines.
  • The synchronization don’t disable interrupts, that’s a cool feature made with a global system lock (a counter) that prevents interrupt treatment to change system global variables or switch threads.
  • Delegation on interrupt service routines, when an interrupt treatment is heavy, it’s possible to delegate that work to system defined thread, by doing so, the system is “always” susceptible to interrupts.

Final Comments

The MOS source code is available on github and I’ve created a new branch for the AVR32 port, in here.
This operating system is my way to apply all the concepts that I learned studying embedded and PC OSs and migrate my software design skills from managed to unmanaged.

AVR32 First Discoveries

I started my study on AVR32 architecture, I’m amazed, angry and confused on what I discovered in this short period of time, the typical when your accommodate to awesome ARM architecture.

The development environment for the AVR32 is the infamous AVR Studio 5.0, based on Visual Studio (probably the best IDE that I’ve ever worked with) and have some cool and annoying features. The best feature of AVR Studio is the Visual Assist X plugin, basically that’s the best IntelliSense that I’ve ever used for C/C++ development. The most ridiculous thing that AVR Studio 5 have is claiming to have C++ support because that’s a total lie, when you download there is no C++ support, you have to download a buggy plugin if you want the support.

A cool feature of this IDE is the “Eclipse Like” toolchain settings editor, the problem with this? They removed the option to define the linker script (To those that don’t know, when you don’t define a linker script -T myscript.ld, the linker will use the toolchain defined script), fortunately if you configure in the “other options” the -T switch  with your linker script, it will be used instead of the default.

Nevertheless the AVR Studio 5 IDE seams a very powerful tool if you want build applications for the Atmel boards, but if you want to go deeper you will find some ridiculous bugs and workarounds.

AVR32 Quick Reference for Developers

The AVR32 architecture is divided in two main micro architectures, the AVR32A and the AVR32B. The AVR32A was designed to be cheap, by doing so this microarchitecture doesn’t have a full register bank for interrupt/event treatment. The AVR32B was designed to time sensitive applications, so the interrupts have their own register bank.

NOTE: Throughout this posts I’m only going to talk about AVR32A because the CPU I have is a AT32UC3A3256, AVR32A compatible with floating-point, so every time I refer AVR32 features and explanations I’m talking about AVR32A (for instance the AVR32B boot-loader and memory layout is different from the AVR32A).

Registers

There are 15 registers in this architecture they are:

  • R0-R8, general purpose registers.
  • R9-R12, argument registers.
  • R13 – The stack pointer.
  • R14 – The link register.
  • R15 – The program counter

You are probably asking where are the flags and other runtime information, I’ll talk about it later on this blog.

Call Convention

The AVR32 call convention is the follow:

  • R9 to R12 is used for arguments where R12 is the first argument and the R9 the forth. All other arguments must be passed by stack.
  • r8 to R12 are scratch registers.
  • R0 to R7 are preserved registers.
  • R12 is used as the return value.
  • R13 to r15 “architecture hardware” registers.

Processor Modes

AVR32 has 8 different working modes:

  •  Non Maskable interrupt – system events that can’t be ignored, this is design to events like TLB misses.
  •  Exception – the usual CPU exceptions, data abort, bad instruction,etc.
  •  Interrupt 3-0 – four different interrupt modes, being the interrupt 3 the most priority and interrupt 0 the least.
  •  Supervisor – the system mode, design to support operating systems.
  •  Application – the user mode, the only one that don’t have all the running permissions

On AVR32 when an interrupt/event is detected by the CPU, they store R8-R12 automatically leaving plenty of registers to  handle the interrupt/event.

Status Register

The status register is divided in two parts. In the lower halfword are the common execution flags (Carry, Zero, Overflow, etc). In the highest halfword are information about the mode, the interrupt availability and masks. The status register even have a scratch bit for the application to use if needed, that’s a funny feature. For more information check the Architecture Document at Atmel. Throughout the future blog posts I probably will talk about some specific flags that I’m going to use.

Remarks

  • The stack pointer (R13) is shadowed between Application mode and the system modes (all the other modes).
  • The Status Register is mapped directly on the memory at 0x0.
  • When an event/interrupt is attended the program counter points directly to the current instruction (That’s not true in all architectures, for instance in ARM the PC is one instruction ahead).
  • One cool feature of the AVR32 event treatment is the indirect exception vector, with that you can define your handler where you want (In ARM this handlers have a “fixed” memory location).

Final Comments

The AVR32 documentation is just horrible, there is little information about the “behind the scenes” I had to reverse engineer the call convention and that’s just ridiculous. If they are trying to beat ARM, they must explain what they were doing, they must get the attention of people who want to work with the hardware as much the people who make the applications to run on it.

I see interesting stuff, like the system registers, the four different interrupt modes and the development environment(Visual Studio + Visual Assist X = FTW) but I don’t feel that they care a lot about the developers, they don’t care about the people that want to work just above the hardware, “talking” to it and configuring it. From an application designer view, AVR32 is awesome, they give you all the device drivers and interfaces to talk to the hardware (something that is on going on the new ARM Cortex convention) and with Visual Studio the only difference between coding for PC or AVR32 is that USB JTAG cable connected to your PC.

References

AVR32 Architecture Manual

32-bit AVR UC3 Application Note

%d bloggers like this: