Week 8 - Embedded Programming

Assignment

Individual assignment

read a microcontroller data sheet

program your board to do something, with as many different programming languages and programming environments as possible

Group assignment

compare the performance and development workflows for other architectures

Link to group page

Process

The datasheet conundrum

The very first step to complete this week assignment was indeed to understand what a data sheet is and how to read it. I found this particurarly important because inside a data sheet, notwithstanding the strict engineerese language, you can find answers to a lot of problems if you know how to understand them.

After retrieving the datasheet of the microcontroller I installed on my board during Week 6 assignment - an ATtiny44A - from the official website I found out that there are two way to read a datasheet:

• Read it through and through, from page 1 to page 286, experiencing migraine and nosebleed after just few lines of the extremely technical exposition;

• Read only the parts you’re interested in for solving a particular problem you’re stuck into, but risking either to fall in a downward spiral of reference jumps between pages or not really understanding what you’re reading about because you’re too slack to delve into terms and aspects you don’t know.

Somehow I adopted both solutions, meaning that at first I did a rapid skimming of all the document, slowing on the most interesting or relevant parts, so that I could have an overall knowledge of the composition of the datasheet. Later on the assignment I would often get back to the datasheet to check specific chapters and sections to understand things I wasn’t grasping or to confirm some guessworks.

I regret a little not having a better knowledge of microcontrollers and electronics, that in fact resulted in a lot of doubts and headscratching about what I was reading, but I don’t regret stating that most of the information you might need about a microcontroller are in plain sight in the first pages of a datasheet. These alone can give all you need to kickstart your quest in the muddy word of embedded programming.

Getting a toolchain together

A really important task I completed before properly starting to code was to set up my environment in the most optimal way, with all the required softwares and tools that could make me not lose time while coding and debugging (this was indeed a good proposal, but I was really naive to think that I could guess all the issues forefront. More on that later).

As wrote, during the week 6 module I made a copy of the Hello World board, while in the Week 4 module I did a FabTiny*ISP programmer, making it all the necessary hardware for this week.

On the software side, since I own a Macbook I opted to install Crosspack. It’s a well-documented OS bundle that has in itself both the softwares avr-gcc for compiling and avrdude for flashing, and comes very in handy especially when you have to deal with many projects. Essentially, it resumes in few functions all the commands necessary for embedded programming and offers even more features to organize such projects. For example, via the Terminal app, you can create a new project writing the command avr-project along with the project name.

As seen in the picture above, this command automatically creates a folder hierarchy, a main.cfile, a Makefile with many functions already prepared and an Xcode file in case the user wants to make use of this IDE. All this features can be set and organized in templates that can ease furthermore the process of creating a functional environment for programming.

Due to some issues occurred during my experiments (again, more on that later), I also had to set up an alternate environment composed of the Arduino IDE and an Arduino Uno board which I used as integration or substitute of the other elements of the toolchain.

Using an Arduino as a programmer is a rather simple operation. You simply have to plug your board to the PC, turn on the Arduino IDE and select a few parameters. These are:

• Setting proper Board, Port and from the Tools menu (in my case Arduino/Genuino for the former and /dev/cu.usbmodem411 for the latter). This options will let the IDE know to who and to where communicate;

• Selecting the Examples -> 11. ArduinoISP -> ArduinoISP command from the File menu;

The last operation will open up a new sketch with all the necessary code to flash to the Arduino so to make a proper programmer (the ISP pinout is shown in the comment section). Once the code is loaded, the Arduino can be used both via the Arduino IDE and avrdude.

To make avrdude and Arduino cooperate, all that is needed is to set up properly the Makefile (to avrdude, Arduino behaves like an avrisp and is recognized as such), instead to load code through the Arduino IDE all that is necessary is to press SHIFT while pushing the load button in the IDE interface. Doing so will make the programmer the medium to the programmee and the IDE won’t override its code.

It is also possible to boot code from the Arduino IDE inside a controler through a non-Arduino programmer. To do so, first of all you have to be select the programmer you’re going to use in the Tools -> Programmer command menu, the you have to sure that the IDE is able to recognize the controller you’re going to program and this happens if it has its specifications in its library. If not, you can update the Arduino board library via the Arduino -> Preferences command menu, where at the entry Extra URL for Board Manager you can add links to json files where not-known boards are defined.

For example, to load into an ATtiny44 as I did, before I had to prompt a link retrieved from the Arduino Playground website were specifications for many boards of many brands are collected and documented.

After adding the micro-controller to the Arduino Board Manager, you have to select its settings in the same way I did for the Arduino board when I flashed it as a programmer. From the Tools menu, you have to choose the proper Board and Port settings, along with CPU and Clock as defined in your project.

In my tests, I did all these permutations due to some very frustrating and recurring issues (again: much more on that later).

Even though my work environment could be complete in this way, there is a couple more tools that I installed to help me deal with the assignment more efficiently.

The first one is nothing but a simple alias-like script for bash that I found on GitHub made by user jlhonora called lsusb. It simply replicates the function of the lsusb command in Linux, that shows in your Terminal window what is connected to your computer buses. It’s not really functional but it helped me in different situations to understand if I was having connectivity problems with my boards.

Link to the lsusb GitHub repository

Another really helpful tools I used is Serial by software house Decisive Tactics, which enables a stable and simple communication with the serial ports of your computer to anything connected to them with a GUI. This is not a free software, but the 7-day trial period was just enough to complete my assignment.

I installed this software because I wanted to try some serial communication, but this task would have never been accomplished without the very precious help of a handy ASCII table with serial control characters from a Sparkfun tutorial and a Sparkfun FTDI SmartBasic which I used to connect the USB port of my computer to the FTDI connector of my board, unfortunetaly with a lot of failures in the process (yeah… expect that later).

Testing a simple program

As mentioned above, I already tested the functioning of my board during a previous assignment, but repeating some of these tasks was helpful to ensure a more understanding of both ATtiny controller and C language. Furthermore I was able to properly understand a design flaw I committed in my board that I was able to work out.

Essentially, when I added a button and an LED to my board I erroneously put a the button in between one of the controller pins and the VCC trace. This is generally considered not a good practice in the world of electronic boards, since it leaves a microcontroller pin subject to voltage fluctuations that might occur during when the board is on. A much better positioning would have seen the button connected to the ground with its corresponding microcontroller pin set steadily to 1, so that when the button is pressed the pin would directly go to 0. This system leaves to no doubt, while my setting needs some adjustments.

I found two ways to solve this. The first one involves a switch with a ground terminal, but this solution still requires some initial design decisions. The second one - which I adopted - works solely on the programming side and prevents you from adopting any hardware modification.

What I did was to set up the pin connected to the switch as an output pin. This setting activate an internal pull-up resistor that prevents floats, then I wrote my code still considering the pin as an input (meaning that I used all usual input reading commands, since the compiler makes you use them regardless of the pin I/O setting).

I tested this solution writing a little code in C involving the AVR/io.h library that makes you light up the LED when the button is pressed.

C program

And as proof of work I wrote the same code with the Arduino libraries and tested it once again.

C program with Arduino libraries

Both programs worked and I also checked in the datasheet to find some kind of confirmation of the controller behaviour that I supposed to happen, which I found at page 54 in the section 10.1.1.

Few notes on avrdude and Makefiles

While working on the Arduine IDE simplifies the process of flashing code inside a board to the stroke of a key (and it’s something very, very useful especially when you work on a thight schedule or don’t want to delve too much into the details of a microcontroller functioning), I found it very interesting and satisfying to practice with all the possibilities offered by working with avrdude.

Aside the fact that this way gives a much deeper understanding of how a microcontroller works, it helped me a lot to study and analyze how a makefile works in order to the a better use of my board. Still, getting to understand a makefile without any instruction can be cumbersome, as well as customizing one without knowing how it really works.

Crosspack both ease and harden this process by generating a brand new makefile for any new project. It helps in many ways, because inside it you can find ready-made the commands necessary to flash both fuses and program inside your microcontroller, but in a way it also hinders because most of the commands are written to a rather high abstraction level for a beginner like me and in its default setting it’s very improbable that avrdude can work just fine with your board (in fact, without customization avrdude thinks you’re about to program an ATMega8 controller with an stk500v2 programmer).

So, to properly customize my makefile, I had to pass a considerable time in studying all its components.

What I essentially learned is that a makefile is nothing more than a text file in which are defined a series of parameter and commands that you set up accordingly your programmer and your microcontroller and that lets you properly communicate and boot code. There are many things to take in consideration when setting up a makefile, such as the controller clock speed, the files to load and the devices used, but the most difficult part that I encoutered regarded the flashing of the microcontroller fuses.

The fuses are a set of registers inside the microcontroller that to a very low level define its behaviour with many, many ramifications (clock settings, memory management, commands priority and so on). The ATtiny44 has three 1-byte large fuses that are set with logical 0 and 1 usually with a hexadecimal values, that have to be defined before you even start to code anything inside the controller (if you have fuses set up improperly your code can be faulty and unpredicatable, which happened to me and - again - will tell more about later).

It seems and indeed is one of the most head-scratching topic in the world of embedded programming, and even a quick read of the datasheet can be a confusing experience (pg. 159 - section 19.2). And although a full read of this specification is necessary in order to understand what we’re talking about, usually programs such as avrdude already have commands for standard fuse settings and there are in existence many resources that ease the pain of finding a proper fuse setting. A very handy tool that I used is eleccelerator’s fuse calculator, that boast many boards configurations and explain plainly and fully every fuse set up.

Getting back to the specifications of makefiles, be it for fuse or code flashing, to the core what you do inside a makefile is to specify automated avrdude commands. As I wrote, Crosspack alread features many commands for which you only have to change the few parameters.

For example, the following lines describe a command that activate certain fuses inside an ATmega8 controller programmed with an AVR Doper ISP programmer:

DEVICE     = atmega8
CLOCK      = 8000000
PROGRAMMER = -c stk500v2 -P avrdoper
OBJECTS    = main.o
FUSES      = -U hfuse:w:0xd9:m -U lfuse:w:0x24:m

AVRDUDE = avrdude $(PROGRAMMER) -p $(DEVICE)

fuse:
	$(AVRDUDE) $(FUSES)

Prompting in Terminal the command avrdude fuse while in the same folder of the makefile would mean flash the fuses of the controller in the same that would have done the more verbose and error-prone command avrdude -c stk500v2 -P avrdoper -p atmega8 -U hfuse:w:0xd9:m -U lfuse:w:0x24:m.

So if you wanted to use the same command for a different controller - i.e. an ATTiny44 - all you had to do was to change the parameter DEVICE inside the makefile and then you could use the very same command for the same operation.

Another way to use makefile for certain purposes is to define as many commands you need for all your different situations, like having two or more commands for two or more specific controllers, programmers, files to load and so on.

That’s is exactly what Neil Gershenfeld coded in its makefile for the Hello Board project, where he set up different commands for as many programmers as he devised he (or others would have used). They include programmers such as AVRSP, USBtinyISP, Dragon and Atmel-ICE and more.

While it’s true that you can write a makefile that includes as many options and commands as possible, in my case both Crosspack’s and Neil’s ones are a bit of an overkill and in the end I prefered writing my own so that I could learn better how to set up a makefile and list only the commands that would be the most helpful for my project. For the purpose of this assignment I coded just few commands that would allow me to flash interchangeably with tinyUSB and Arduino programmers. This is done not only by switching the programmers parameters, but also assigning different ports for communicating. The commands definitions are written like this:

flash-usbtiny: $(PROJECT).hex
	avrdude -p t44 -P usb -c usbtiny -U flash:w:$(PROJECT).c.hex

flash-arduinoisp: $(PROJECT).hex
	avrdude -p t44 -P /dev/tty.usbmodem411 -c avrisp -b 19200 -U flash:w:$(PROJECT).c.hex

Inside these commands it’s invoked also another command, $(PROJECT).hex, which compiles to code to be flashed and generates a machine language version readable for the microcontroller.

(Almost) Designing a new program

To properly test the capabilities of my board, I devised a simple program that takes into account the button, the LED, some internal functions of the controller and the serial port connected to it.

The idea was to simulate a simplified version of a Finite State Machine that will be part of my final project.

I wanted a machine that worked on three states:

• Input state: you can feed the board with inputs from the keyboard. If they are numbers they are accepted and added to a container, otherwise they are discarded;

• Evaluation state: at the press of the board swithc, the inputs are no more accepted and the CPU would check the sum of the values prompted against a const value (along with a tolerance range). If the sum is between the range the evaluation is positive, otherwise it’s negative;

• Result state: If the evaluation is positive the board LED turns on, otherwise it stays off. At the press of the board button, the machine resest and start back to the input state.

Aside from the logical components, to complete these program I had to deal with many different technical issues that I mostly solved by analyzing (read: coping) Neil Gershenfeld example code, specifically the echo program that involved the use of serial communication (the most difficult part to grasp for me).

Initially, to do so, I opened side by side my own code and Neil’s one, so that I could directly see what was working on one side and could be used for the other.

This might not be the most honest way of working, but it was really helpful for me especially in the most technical parts where little documentation was provided and a lot of guesswork often took me nowhere.

Serial communication

The board was very conveniently provided with an FTDI port that could make it communicate with anything else with a serial communication protocol. In my case I wanted the board to accept and send with a computer, which could be used also as a report tool. The first step in that direction was to set up the hardware, which I did with the help of either with the SparkFun Smart Basic mentioned before or an FTDI to USB cable.

Both do the job just fine, except for a simple issue that encountered with the SparkFun tool that I wasn’t able to solve without a lot of stress (and of which - you already know - I’ll talk more about later).

With all the tools in place, the first thing I did was to try Neil’s code on my board, which simply sends back in the serial port any character you prompted to the board.

After loading the program and setting up the serial communication (the aforementioned program Serial makes this job very easy and intuitive, you just have to plug-in and you’re at it), the code worked just fine except for a formatting issue of the messages. This happened because in the code was missing the return carriage character to prompt to the serial port once the message ended. To fix this I just added the line put_char(&serial_port, serial_pin_out, 13); at the end of the main loop and the result was much more readable.

After dissecting Neil’s code, I took advantage of all its communication functions and adapted them for my purpose. In the end, I divided code in many different functions so to have a more precise response from the different stages and not relying too much on technical aspects which are not really my best. This has resulted in separate and useful functions for tasks such as:

• Activating/deactivating the LED;

• Activating/deactivating the LED for a time span;

• Validate the character in input (non-digits were discarded);

• Sending messages through the serial port;

• Checking the switch status;

Even though, with clear instructions in mind, all the functionalities of the program could be accessed through the electronic board and a keyboard, I coded few minor incoming message functions that could help understand better what was going on and provide instructions for debugging.

After many, many loops of the design-test-debug cycle (I promise we’re close to it, but more on that later) I finally had my program ready to be flashed.

HELL YEAH!

Issues, troubles and bugs

The following is a list of the problems I’ve met during this assignment. Few of the are just signs of bad luck, other of misunderstading of my tools, many other because of my own bad practices. I listed them in the hope that other people won’t fall in them again.

• I was unlucky enough to get both my board and my programmer burned during this week’s assignment. And I was even more unlucky to not figure that out very rapidly, hence trying to figure out what was going on every time I received a bad response from avrdude. A bell should have rang, if the controller couldn’t get a hold of its own fuses… Only when eventually a pillar of smoke rose up from my programmer that I figured that something bad went wrong with it (in total disclosure, I actually almost burned myself touching it to get tha realization) and chose to try for different solutions. For such, I went through a lot of configurations using borrowed programmers and Arduino boards. Having a plan b is always a good practice, especially if you suspect that your boards might have issues, but in my own case, specifically when using the Arduinos as programmers, I had to deal with a bit of widespread misinformation regarding the wiring of the board. It’s widely believed that to use an Arduino board as a programmer it’s necessary to use a polarized capacitor connected from the Reset pin to the ground. This is also confirmed by official Fab Academy tutorial but actually this tip got me into a lot of guessworking. In my case, adding a capacitor to the setting wasn’t make it work and I thought for a long while that it was fault of something on your board. Actually, for sheer serendipity, I found out on an Arduino message board (can’t find the link anymore!) that the need of a capacitor is true only for Rev 1 and Rev 2 boards, for which I found confirmation on this comparison article, and since all the boards I used where Rev 3, in fact when I didn’t use the capacitor at all the makeshift programmer finally worked fine! I won’t say then that tutorials aren’t useful, but you always should question yourself if they could really be applied to your specific case.

• Speaking of Arduino as ISP programmer, I took me a lot to figure out - in my late 2008 Macbook - which was the port it used to communicate. Knowing the programmer port is an essential thing if you use avrdude, and in the Arduino IDE it was possible to see that the port associate to the Arduino as ISP was called /Dev/cu.usbmodem411. What I knew not is that OS systems have this peculiar way of calling the port of the same device in two different ways, but still you have to know the right one to use if you want your makefile to work. Otherwise, you’ll get an error report like the following. Once the address /dev/cu.usbmodem411 was replaced with /dev/tty.usbmodem411 the programmer worked just fine. I figured out the mistake because I could see in other people’s setting that they always used to call a /dev/tty.* port and because if when I prompted the bash command ls /dev/* to see all the active ports in my computer I could see the .cu and .tty couple appear and disappear at the same time whenever I plugged and unplugged my programmer.

• Speaking of Arduino IDE, microcontroller pinouts are something that no one speaks about unless you ask precisely for them, which is some kind of a paradox since you can’t really ask about something you don’t even know it exists in the first place. However, when you have to code for a microcontroller inside the Arduino IDE you always look for its related pinout mapping: that describes how the controller pins are eunmerated inside the IDE, which rarely (never) corresponds to the datasheet enumeration. Not knowing this could make you spend hours not figuring out that in your code you’re calling the wrong pins for no apparent reason.

• In the occasions where my code worked fine I was still questioning something I knew was flawed but didn’t know how. Specifically, I could see that when I had an LED set on or off through code for a given time, it would always be for longer than expected (approximately more than twice). This issue got back in my mind when I calculated the fuses and found out that there are many settings for providing external or internal clock signal. It was then that I could see that Neil’s clock fuses are always set to the internal clock (8 Mhz for the ATtiny44). When I set up the fuse so that the clock had to rely to the external resonator I could finally experience a much more reliable timing from my LED blinks. How is it possible that Neil relied to this faulty fuse setting? The answer to this question came out when I analyzed Neil’s code and found out that he provides a function to scale the clock fuses inside the code itself. Specifically, these are the lines of code involved, which set the clock divider to 1:

   CLKPR = (1 << CLKPCE);
   CLKPR = (0 << CLKPS3) | (0 << CLKPS2) | (0 << CLKPS1) | (0 << CLKPS0);

• The Sparkfun Smart Basic was a very helpful tool to my board running and communicating at the same. Still, it gave me problems when I unplugged the ISP chord, like it was turned off. Upon some inspection, I found out that the Smart Basic had a jumper soldered that made it work only on 3.3v making it useless for my board.

• Being a total agnostic of serial communication I found myself in a lot of dead ends when dealing with the most technical aspects, and the most non-trivial might have been the Baud rate. This value represents the speed at which data travels, usually in UART lines, expressed in bps. A comment in Neil’s code gives for granted to use 115200 Baud rate for the serial communication, but how he calculated this value was at first was very misterious for me. Upon some inspection of the code, the only value I found onto which the commmunication relied was a directive called bit_delay_time set to 8.5 microseconds. Further, in the tx and rx functions I could see that every data chunk was composed of 10 bits (1 start bit, 8 data bit, 1 stop bit). These values were very interesting and confirmed what I could read online on a Sparkfun Baud Rate tutorial, but still I wasn’t getting where derived that 8.5 value. I opted to do some math and actually I found out that 1 (second) / (115200 (Baud rate) / 10 (data frame)) is almost 8.6 microseconds. Considering a reasonable time tolerance (1-2%), this was how the bit delay was calculated and choosing such Baud rate is perhaps nothing more than a standard setting based on the average size of words and pages sent over serial communication. In any case, messing even a little with such parameters (or not setting properly the configuration of the serial monitor) can result in very fascinating but totally useless screens like the following:

Result

It was not an easy journey, but in the end the result was exactly what I wanted to achieve.

My board was able to discern keyboard input, discard non-digit values, react to the button pressing, flash the LED accordingly and properly communicate through the serial port.

The following are the link to the source code and avrdude’s makefile:

Main.c file

Makefile file

Gist & Further development

The Arduino IDE seems to simplify and ease a lot the whole process of setting up your working environment, but after some use and comparison with other processes it looks like it hides a little bit too much under the hood and if you prefer to have more customizations and control over the many small details of this topic, perhaps it’s better to leave for smaller and less demanding projects.

Having Neil’s C functions at hand was the most useful way to learn how the blurred line between software and hardware works, but still - especially for the final project’s scope - I know I’ll have to figure out by myself a lot of underlying mechanics to properly understand embedded programming without using vast and pre-compiled libraries.

Tools and software used

• Sublime Text 2
• Crosspack
• Arduino Uno
• Arduino IDE
• Serial
• SparkFun FTDI SmartBasic