Friday, July 27, 2018

Kinetis FTM, DMA, and Stepper Motors

This blog will be the first of a series taking a deep dive into the NXP Kinetis Flexible Timer Module, and how to use DMA to run stepper motors and read encoders.

[Working knowledge of Kinetis MCU and DMA controllers is required. Working knowledge of stepper motors, at a high level, is required. Understanding C and pointers is a must.]

The Kinetis K22 Reference Manual devotes over 130 pages to the Flexible Timer Module, FTM.
There is a lot of documentation on using the FTM for brushless DC motors, BLDC, and PWM, but little, if any, on stepper motors.

NXP also provides an Eclipse based IDE, MCUxpresso 10.2.1 as well as an SDK, 2.4.1. The SDK provides an API for configuring and using peripherals, like the FTM, PIT and DMA. The 2.4.1 SDK will be used in code examples here. Downloading the SDK does require registering at the NXP website (free, as in beer) and requesting a specific Kinetis hardware configuration, either an evaluation board or a specific MCU. The blog uses the SDK API in code examples.

A stepper motor is an industrial standard motor that provides the ability to move a step with just an single digital pin. The physical motion is more complicated, but from the view of the MCU a stepper motor is moved by asserting and de-asserting an output pin.

The STEP signal rotates the stepper motion is fixed angle, typically 1/200th of a revolution. Attaching the shaft of the motor to something that moves in a linear fashion results in the linear or positional motion.

A stepper control chip, like an A4988, provides the simple STEP signal interface to a stepper motor.

A stepper motor is 'turned' by asserting the STEP signal. The STEP signal has to be de-asserted before the next step. The STEP signal has to be asserted for some minimum amount of time, which depends on the stepper control chip, but typically in the microsecond range or less.

The ENABLE signal needs to be set to 0 and the DIR signal set to CW or CCW direction. These signals do not change while the motor is turning.


1 - Set ENABLE low.
2 - Set DIR CW or CCW
3 - Assert STEP
4 - Wait a specific time
5 - De-assert STEP
6 - Wait a specific time
7 - Repeat 4-7 until all STEPs have been asserted.

The STEP signal is a square wave, whose frequency equals the steps per second. Assert the STEP pin, at a fixed frequency, and the stepper motor rotates at a fixed RPM. Vary the frequency from a longer interval to a shorter interval, to a constant interval, finally to a longer interval is the sequence used to move to a position.

[Insert constant and variable square wave diagram here.]

The step motor data is a table of step pulse intervals between STEP pin assertions. There are many ways to generate the square wave, given a step pulse interval table.
There are many, many algorithms for generating this table of time intervals.

This blog discusses how to read this step table to generate motion using the STEP signal and the timer intervals in the table. It does not discuss how to generate the table.

MCU Intensive Stepper Motion

The simplest method to generate a square wave for stepper motion is the following pseudo code.

Generate a table of the time intervals for the STEPs required to generate the desired motion.

For each element, i, in the step pulse table

    Assert the STEP pin.

    Wait 1/2 ith time interval

    De-assert the STEP pin

    Wait 1/2 ith time interval

The above pseudo code generates a 50% duty cycle square wave. Half the time the STEP pin is asserted, half the time the STEP pin is de-asserted. The 50% duty cycle represents the simplest square wave to generate. The algorithm to generate the step pulse table could generate some other percentage duty cycle. The pseudo code for a duty cycle other than 50% might look like this.

Generate a table of the time intervals for the STEPs required to move the desired distance, with a fixed assert time, two entries per step.

For each pair of elements in the step pulse table

    Assert the STEP pin.

    Wait the ith time interval

    De-assert the STEP pin

    Wait ith + 1 time interval

The basic algorithm is the same.

Both of the above algorithms are MCU intensive. The MCU can only perform the task of asserting and de-asserting the STEP pin. This is very inefficient. It blocks the MCU from performing any other tasks during the motion.

Programmable Interrupt Timer

A programmable interrupt timer, PIT, can be used to increase the efficiency of asserting the STEP signal.

A PIT can be used with the following two features. The first feature is the PIT counts down from a value to zero. Secondly, when the PIT reaches zero it generates an interrupt.

The pseudo code below describes the stepper motion using a PIT.

Generate a table of the time intervals for the STEPs required to move the desired distance, one entry per step.

Initialize the PIT. Do not start it.

Read the first time interval from the table.

Set the PIT count down time to 1/2 the time interval.

Set pulse_state to assert.

Assert the STEP pin.

Enable the PIT interrupt.

Wait for the PIT interrupt to indicate the motion is done.

  Clear the PIT interrupt.

  If pluse_state is assert
    De-assert the STEP pin.
    Set the PIT countdown timer to 1/2 the time interval.
    Set pulse_state to de-assert.
    Get the next time interval.
    Determine if motion is done.
    If motion is not done
      Assert the STEP pin.
      Set the PIT countdown timer to 1/2 the time interval.
      Set pulse_state to assert.

The PIT pseudo code provides more efficiency by not waiting for the time to expire for each step. The PIT interrupts the MCU for each assert and de-assert. The MCU is free to perform other tasks during the time to the next step.

If there is only one motor and the motor does not move very fast, this algorithm is sufficient.

There is a drawback. For each step there are two interrupts. An interrupt is expensive. There are registers to save and restore and an interrupt to clear.

Stepper motors typically have 200 steps per revolution. If the stepper motor is turning a shaft where for each revolution the shaft moves linearly 1 centimeter and it is desired to move at a maximum velocity of 20 centimeters per second, this is 200 * 20 or 4000 steps per second.

4000 steps per second is 8000 PIT interrupts.

The exact number of interrupts of any motion will vary, but the number of interrupts will typically approach thousands per second.

This many interrupts affects the MCU's ability to perform other tasks. Again, if the motion is short, or the motion is slow, the number of interrupts is not an issue.

Flexible Timer Module

The Flexible Timer Module, FTM, provided in the Kinetis processors, combines the function of a PIT with the assertion or deassertion of a pin.

The FTM has a master clock and the assertion and deassertion of the STEP pin is handled by a channel of the FTM. There are several channels available.

The FTM has many other features, but these are the features that will be used to output the STEP signal.

FTM Clock and Timer

Each FTM module has several clock modes. To use the FTM with a STEP output, system clock mode is selected. This timer is the master timer for all channels. CLKS=01 sets the clock as the system clock.

The timer has several counting modes. The mode selected for STEP output is the count up timer.
QUADEN = 0 and CPWMS = 0 sets the count up mode.

The timer can have a tick interval the same as the Kinetis bus clock, or any power of 2 division up to 128. PS sets the prescaler value. [Note: Kinetis documentation uses the term system clock. It should say bus clock.]

The timer can be configured to count the full 16-bits or some smaller value. MOD sets the maximum count-up value. CNTIN sets the lower value of COUNT. For STEP output, set CNTIN to 0 and MOD to 0xFFFF. This provides a full 16-bit of FTM timer interval between STEPs. A longer interval can be created. That technique is discussed later.

The FTM timer is set to be a free running timer, never stopping when FTMEN=0 and MOD=0xFFFF.

In summary, to set a 16-bit count-up free running timer, CLKS=01, FTMEN=0, QUADEN=0, CPWMS=0, CNTIN=0, and MOD=0xFFFF.

FTM Interrupt

The FTM has one interrupt for any event. Which FTM event occurred has to be determined by examining status registers. Each channel has an interrupt enable bit and an interrupt status bit. Other interrupts, such as faults can be configured.

It is not appropriate to set the TOF overflow interrupt bit with the free running clock as it will constantly overflow. It could be used as a low speed interrupt.

FTM Channel

To assert or de-assert the STEP pin, a specific FTM channel is assigned to a specific output port pin using the port configuration of the Kinetis MCU.

The FTM channel is assigned a timer value, CnV. When the timer of the FTM associated with the channel is the same value as the FTM channel value, the pin output action occurs. This is called output compare mode.

The pin output action can be assert, de-assert, or toggle.

The FTM channel can assert an interrupt event at the same time, if enabled.

Output Compare Value

Each FTM channel has an output compare value, CnV. This is a 16-bit value.

The output compare value is not the step time interval. It is the last output compare value plus the step time interval. The output compare event occurs when the output compare value and FTM COUNT are equal.

The FTM clock is a free running clock, not a countdown clock. The compare value is absolute FTM clock time to assert or deassert the STEP pin.

Each step time interval has to be added to the previous output compare value to set the next output compare value.

To start, the first output compare value is a few counts greater than than the current FTM timer, COUNT.

FTM and Assert and Deassert STEP Pin

In each FTM channel is a command and status register, CnSC. This is the register that states what the STEP pin should be when the output compare value matches. This register sets the assert, de-assert, or toggle action.

This register is also where the interrupt enable bit is set when the output compare match occurs.

FTM and Motion

The FTM timer acts as a PIT that also changes the state of a pin.

Generate a table of the time intervals for the pulses required to move the desired distance, one entry per step.

Initialize the FTM. Do not start it.

Read the first time interval from the table.

Select an FTM channel.

Enable the FTM interrupt.

Set FTM channel CnSC to assert the STEP pin.
Set FTM channel CnV a few FTM time ticks from COUNT, to create the first FTM interrupt.

Wait for the FTM interrupt to indicate the motion is done.

  Clear the FTM interrupt.

  [Determine which channel interrupted, if more than one channel]

  If FTM Channel CnSC is assert
    Set FTM channel CnV output compare to 1/2 the time interval.
    Set FTM channel CnSC to de-assert STEP pin at end of time.
    Get the next time interval.
    Determine if motion is done
    if motion in not done
      Set FTM channel CnV output compare to 1/2 the time interval.
      Set FTM channel CnSC to assert STEP pin at end of time.

A bit more setup to start the motion, and two FTM settings per interrupt. This is the same as the PIT, one setting for the STEP pin and one for the time.

FTM doesn't seem to improve the MCU efficiency that much for the setup overhead. Why use it?

Using Direct Memory Access, DMA, the FTM can be significantly more efficient, hardly any MCU time. Setting up the FTM to use DMA for stepper motion is the 'trick'.

[Link to next blog.]

Thursday, July 19, 2018

Primes and Sines

Primes and Sines

Prime numbers are a fascinating set of numbers. A prime is a number divisible by only one and itself. 11 is prime. 6 is not prime.

Primes are a subset of the counting numbers, 1, 2, 3, 4,...  Prime numbers are never negative. The first few primes are 2,3,5,7,11.

1 is not considered a prime. 2 is sometimes not considered a prime. Math papers on primes might start 'Consider the set of primes starting at 3...'.

When a number is not prime it has a list of prime factors. This is part of the Fundamental Theorem of Arthimetic. The non-prime number is created by multiplying all the prime factors together. A prime factor may repeat. 18 has 2, 3, and 3 as prime factors (2*3*3=18).

Any even number is a multiple of 2. All primes numbers are odd, except 2. This is why 2 is sometimes left off the list of primes numbers.

To determine if a number, N, is prime, a simple sequence of steps can be followed.

1 - Find the list of all primes less than the square root of the number N.

2 - For each prime, p[i], in the list of primes, N/p[i].
2a - If there is no remainder from the division of any p[i], then N is not prime.

Given 21, the square root of 21 is 4. The list of prime numbers less than or equal to 4 is 2 and 3. 21/2 = 10, with 1 remainder. 21/3 = 7, with no remainder. 21 is not prime.

To create a list of primes, one follows a sequence called a prime number sieve.

First select a maximum number to be examined, say 1000.

2 is given as a prime (or ignored).

Create a list of 1000 numbers, mark all numbers as prime.

Starting with 3, add 3 to 3. In the list, at 6, mark 6 as not prime. Continue adding 3 to obtain the next multiple of 3 marking that entry in the list as not prime until the maximum number is reached, 1000.

Repeat for all odd numbers.

Examine each odd number in the list. Those entries still marked prime is the list of prime numbers less than 1000.

This is prime number sieve is very easy to write in computer code. There are various methods of optimizing the method. One such method is found in GMP. The GMP prime number sieve finds all the primes in the first ten million digits is much less than one second (running on a recent laptop).

The basic process is 1) create a list, assume a numbers are prime, 2) mark those numbers not prime, 3) examine the resulting list looking for prime numbers.


A sine wave is a periodic wave. It repeats as much as desired.

y = sin(x) [1]

generates a periodic wave, starting at 0,0, with four points being (π/2, 1), (π, 0), (3π/2, -1) and (2π, 0). The sine wave cross y=0 at π intervals.

y = sin(x)

If the equation is changed to

y = sin(π/2 * x)  [2]

Equation [2], starting at 0,0 has the following four points (1,1), (2,0), (3,-1), (4,0).

y = sin(π/2 * x)

For equation [2] the value of y is 0 at every multiple of 2. Equation [2] can be viewed as marking all those numbers that are multiples of 2 and thus not prime.

y = sin(π/3 * x)  [3]

y = sin(π/3 * x)
For equation [3] the value of y is 0 at every multiple of 3. Again all those numbers that are multiples of three have y=0 and visually mark the number as not prime.

Now if equations [2] and [3] are shown together.
y = sin(π/2 * x) and y = sin(π/3 * x)

Any odd number not crossed by equation [2] or [3] is prime. 5, 7, and 11 are the next 3 prime numbers.

Any point where y=0 for either equation [2] or [3] represents a number that is not prime.

This is true for all primes up to 24. Why 24?

Any odd number less than 24 has at least 2 or 3 as a prime factor. 25 is the first number without 2 or 3 as prime number, 5 * 5.

The list of prime numbers after 3 and before 24 is 5, 7, 11, 13, 17, 19, and 23. Using just 2 prime numbers we find an additional 7 prime numbers.

y = sin(π/2 * x) and y = sin(π/3 * x)

If we add a new equation

y = sin(π/5 * x)  [4]

y = sin(π/2 * x), y = sin(π/3 * x), and y = sin(π/5 * x)

It is a bit more difficult to see those numbers that are prime. 29, 31, 37, etc. are prime.

This is a graphical representation of the prime number sieve.

Take the absolute value of [2].

y = | sin(π/2 * x) |  [5]

y = | sin(π/2 * x) | 

Repeat this process for equations [3] and [4].

y = | sin(π/3 * x) |  [6]
y = | sin(π/4 * x) |  [7]

Combine [5], [6], and [7].

y = | sin(π/2 * x) | , y = | sin(π/3 * x) | , and y = | sin(π/5 * x) | 
The numbers that are prime are now much easier to identify. The prime numbers are where the three equations lack a y=0 point.

Using [5], [6], and [7] will find the prime numbers up to 48, as 49 is 7*7. The first number without 2, 3, or 5 as a factor. The remaining primes are 7,11,13,17,19,23,29,31,37,41,43,47. With just 3 primes 12 additional primes are found.

This visualization of the prime number sieve uncovered an algorithm that seems more pleasing that the standard prime number sieve, at least to this author.


The sine graphs are visual, but not practical. The curves are continuous. Only the values at odd numbers need to be calculated, but trigonometric functions are expensive to calculate.

It is only required to know if the sine curve is zero or not zero at the point N.

A way of determining if a number is prime is to use the same prime number list, but use modulo arithmetic instead of sines.

Modulo arithmetic was shown above for the number 21.

If a number, N, has a prime sine curve, p, passing through it (y=0) then the modulo of N of p is zero stating that N is not prime.

Given a list of primes, whose largest prime is P, for all odd numbers N from P+2 to (P+2)(P+2)-1, perform modulo arithmetic on N using the list of primes. If all modulo results are non-zero, the number is prime. Testing N can stop when one modulo result is zero.

The graphs above were created using Desmos. The link to the Desmos page with the graphs is here.

Modulo code is found on GitHub.

Saturday, April 28, 2018

Software, Insurance, and Underwriter's Laboratory

Jack Ganssle wrote this article entitled 'Fire Code for Software'.

"Fire code for Software?" is essentially saying why hasn't the software industry developed an Underwriter's Laboratory or an Insurance Institute for Highway Safety for software?

Both UL and IIHS are products of the insurance companies desire to improve human safety, lower claims, and manage insurance costs. UL has been around since 1894. Software has only be around for 50 or so years and only in the last 20 or so years has software been 'mainstream'.

Currently software is not insured. Problems with software are resolved after the fact, usually by litigation.

Regulators, such as the FDA, monitor software for medical devices. The NTSB can audit software when a 'recall' event occurs involving vehicle (car, train, plane) software. Several companies have gone bankrupt because of software mistakes that killed people, Therac-25, for example.

Self-driving cars may push the issue of software testing for insurance reasons.

When the cost of software mistakes are born by someone other than the software builder and the customer, this is insurance, and the insurance companies will protect their interests.

Friday, January 26, 2018

Speculations of an Autonomous Vehicle World

Autonomous vehicles are just around the corner. Tesla has semi-autonomous vehicles for sale today. Waymo, Google, and Uber are all working on fully autonomous vehicles. Delphi, a supplier of auto parts, is selling the sensors necessary for autonomous vehicles to know about their environment.

What will happen to society, jobs, work, travel, and the environment as the autonomous vehicle becomes the norm? This is a speculation on a new world with autonomous vehicles.

In the beginning of the autonomous vehicle world only those early adopters will experience a new life style. Sitting in an autonomous vehicle will be like being in an airplane on the ground. The passenger doesn't care where the vehicle is going, just the vehicle gets to the destination safely and on time.

The world will not have 100% autonomous vehicles soon. It will start slow and build. As it builds, changes will occur in transportation, jobs, and leisure.

The first adoption of autonomous vehicles will be where the most savings of money and labor can be realized.  There will also be adoption by those that can afford new technology, but that will be as a status symbol, a toy. The initial commercial impact will be in long haul trucks, street sweepers, garbage trucks, snowplows, delivery vehicles, pizza delivery, and package delivery.

Already long haul trucks have been driven coast to coast autonomously. [ref] Volvo is advertising a self driving garbage truck. Self driving street sweeper is a direct spin off. Snowplows may be more difficult given the size of the blade and parked cars, but on limited access highways a distinct possibility. Pizza delivery can be by drone, but can also be done by small, go cart like vehicles. Punch in a code to unlock door when vehicle arrives. UPS is looking at deploying autonomous carts from a vehicle to deliver more than one package at a time in a limited area. The UPS truck could drive itself to the deployment location.

Those that drive trucks, short haul and long haul, delivery trucks, garbage trucks, street sweepers, snow plows, or other municipal vehicles will be the first to feel the job stress from autonomous vehicles. Drivers of taxis and buses will be affected as well.

Cities and towns will definitely do a cost analysis of employing autonomous vehicles to reduce costs. When current vehicles have reached their useful life, autonomous vehicles will be available for purchase. Unions will push back. It may take decades to change the union contracts or wait for existing employees to retire. Removing the toll booth operators around NYC was technically feasible when the first FastPass was installed. It wasn't until 2017 that all human operator toll booths were removed.

When an autonomous delivery vehicle arrives with goods for a customer, does the customer unload the vehicle? Does the vehicle unload itself? Is there still a person with the delivery vehicle to unload the goods?

Vacation travel will change. RVs will now be a living room on wheels. One will get to a destination in half the time. The vehicle drives 24 hours a day. Fueling a vehicle will need some thought. A robot gas station perhaps? How will humans tell the autonomous vehicle to pull over for a bathroom break or stop for lunch? Yes, RVs and buses have toilets, but not sedans.

How many less cars will on the road? Will there be less congestion? Will autonomous vehicles drive better resulting in less rush hour delays?

Who will benefit? Fleet owners of autonomous vehicles, body shops that customize interiors, software programmers, advanced auto mechanics, and training schools.

Who will lose? Truck drivers, taxi cab drivers, delivery drivers, public works employees, parking lot owners, parking lot attendees, automobile assembly line workers, car dealerships, auto mechanics, the Teamsters Union. Municipalities will have less revenue because there will be less cars to tax, less parking, so less parking meter fees. The parking meter fees represent a significant percentage of some cities revenues. A law was proposed in Massachusetts that will tax autonomous vehicles for 'cruising' around without parking. The law essentially taxes the vehicle at different rates, with passengers, without passengers, etc. This is an attempt to recapture the lost taxes from less cars.

Ford, GM, and others are looking into subscription service for vehicles, not ownership. Cadillacs are already offers normal cars on a subscription basis. Subscription service is going to be required by the auto manufacturers to make up for lost revenue as the number of cars required drops, by some estimates as much as 75%. [ref].

Everyone will pay a monthly fee to be able to 'hail' an autonomous vehicle. What sort of premium services will be available? Will there be 'congestion' pricing, 'time of day' pricing?

Enter into your 'hail a ride' app how far you want to walk from your home, how far to want to walk to work, when you want to leave and arrive at your destination. A vehicle shows up as requested. It may have no passengers, it may be a van pool, it may be a bus. It all depends on your choices and level of service.

Will there be a bus as we know it? Buses run on fixed routes at fixed times to be predictable. It is used, mainly, by those that don't have vehicles today. Buses won't have to run on fixed routes, if they are autonomous. There may be fixed pick up and drop off points, but the buses don't have to follow a fixed route to get to each point. Either a kiosk or a phone app will be used to say which destination you want to go to and from which location you want to be picked up. Routes will be created dynamically based on the real time demand. Less total buses will be needed and can run 24 hours a day.

The displacement of workers due to work sent overseas took decades and affected mainly manufacturing. Autonomous vehicles will affect the entire transportation infrastructure and all the people that work in it and will be rolled out in less than a decade.

Houses will be built without garages. Housing density go up as off street parking will not be required. What other planning and zoning changes will occur? Will there be a surge in customization of vehicles? Boutique fleet owners will provide unusual, exotic rides for special occasions. The living room sofa ride will now be a reality. Will traveling in luxury take on a new meaning? A TV, Internet, lounger, mini-bar, microwave, refrigerator are all items that can be found in a vehicle today. What will be found in the luxury autonomous car of tomorrow? Multiple day rides in an autonomous vehicles will include a bed. Autonomous vehicles may be fueled or charged while they roll, just like midair refueling of military aircraft. Definitely a time saver.

Will there be an entire generation that doesn't know how to drive? The department of motor vehicles will have less cars to register. You will go to DMV to get a state identification card, not a driver's license.

The autonomous vehicle will be a boon to those who are blind and handicapped, or those who cannot drive for some reason. The mobility of seniors will greatly increase.

The challenges today for autonomous vehicles is snow obscures visibility, snow covers the painted lines. Heavy rain and glare are also an issue. Northern climate residents may see a slower roll out of the autonomous vehicle.

The autonomous vehicle will be assisted by other vehicles and by road side devices to help with navigation. Auto manufactures are designing the communication and devices for a smart highway. The smart highway will help to define the autonomous experience. States may have to do a better job with road line painting for the autonomous vehicle fleet. What happens after a road is paved? There are no lines on the road immediately after paving. Will line painting be autonomous too?

What about computer hacking? Vehicles have already been shown to be hackable. Will a terrorist be able to cause death and mayhem on the road?

What about the classic car owners? How will older cars fit into the world of autonomous vehicles? Will they have to be fitted with special sensors?

The decrease in the number of cars will free up resources for other purposes. Commodity prices will fall as demand for iron and copper to build cars go down. Yes, there will be less cars on the road, but will there be more or less miles traveled? Ride sharing would decrease traveled miles, but those that didn't drive will be more mobile. Does oil demand go down or up?

How will criminals use autonomous vehicles?

The Chinese character for crisis, 危機, combines glyphs from danger, 危險, and opportunity, 機會. The world of autonomous vehicles will be different than today, danger for those supporting the non-autonomous world and opportunity for those that embrace the autonomous world. Will there be room for everyone?

Saturday, September 3, 2016

Loss of the Night Sky

This is not my usual computer geek blog, this blog is on the theme of

"It should be dark at night"

Earth at Night

NASA has published this famous picture of what the dark side of the Earth looks like from space. This picture shows hot bright the cities of the world are at night, and how much of the developed world is lit up 24/7. (A really large version of this map, 10MB, is here. [biggest, 40MB, here.]

How many features can you see of the Earth without sun light? Find the Nile River, the Hawaiian Islands, the border between N and S Korea.

A lot of places 'left the lights on'.

 Light Pollution

All of these lights cause light pollution, light that obscures the night sky.

Here is an interactive map of world light pollution. Light pollution has been put on a color scale. Red is like Times Square, NYC. Black is Point Nemo. Where do you live? Orange, Yellow, maybe Green. [Map from Falchi et. al.]

[Good background, nice tweet, good references for mapping light pollution on the ground.]

Years ago a scale was developed for the apparent magnitude of stars in the night sky. The scale is logarithmic (read the link for all the maths :) ). The edge of human seeing is 6.5 on this scale. Just over 9000 stars can be seen by the human eye if the sky is dark enough (article has a lot more detail on the exact count and where this count is valid).

Add light pollution to obscure the dim stars, and the number of stars you could see at 4th magnitude drops to only HUNDREDs.
Here's a great set of charts showing how the constellation Cynus the Swan changes from 0 magnitude to 6 magnitude viewing.

Excellent demonstration of the difference of two areas with different night pollution. 

Milky Way

Have you ever seen the Milky Way, horizon to horizon? Seeing it makes you understand why the Ancient Greeks called it  gala, 'milk'.

All ancient civilizations have stories on how the Milky Way was created or what it represents. In the modern world, with our night skies, that connection is lost.

Citizen Science, Apps, No Telescope Needed

There is a great mobile phone app "Loss of the Night" (iTunes here). You become a citizen scientist. The program guides you through locating dimmer and dimmer stars until it determines how dark is your sky. It takes 15 or so 'sightings' to get enough data for a reasonably accurate measurement. The authors have a blog, shows the results, good stuff. They are developing a map of how the night sky is changing, literally from the ground up. See this for more details.

Another, more manual, citizen science project that has been running longer than 'Loss of the Night' is Globe at Night. Simple 5 step directions here.

Dark Sky Reserves and Parks

The International Dark Sky Association has started declaring areas a Dark Sky Reserve. It is a rigorous process to get an area designated such a site. Only 11 such sites are listed.

 A lower designation is a Dark Sky Park. The US has many of these, associated with its National Park system.

Finally, a Dark Sky Sanctuary is like a Dark Sky Reserve, but remote, with limited access.

It is a fact of modern times, one must create a park to see the night sky as it was ONLY 100 or so years ago, just outside living memory.

Man Made Lights and Lighting Science

The culprit in the loss of the night sky is the electric light. IDSA does have an excellent page on outdoor lighting and what communities can do to "bring back the night".

One result of light pollution is the brightening of the night sky, Sky Glow.

Another excellent website on lighting is The Lighting Institute at Rensselaer Polytechnic Institute The Lighting Institute conducts a two day course on outdoor lighting, and much more. A great resource for municipal public works departments, planning and zoning committees, as well as departments of transportation at the state level (Yes, USA centric terminology).

Look Up, Measure Your Night Sky, Report It, Get Involved

Lots of links in this article. Lots of people to talk to.

Learn about outdoor lighting.

Learn where you can educate local government officials on what is a loss of a NATURAL RESOURCE, the night sky. They can control lighting issues with local zoning.

Many US states are passing laws regarding the lighting of highways and roads, state buildings, etc.

Bring the Milky Way back, for good!

Friday, July 8, 2016

Software Archaeology

This blog is to discuss what a software engineer, me, has to do when there has been years of neglect to program.

I work in the embedded systems space, so this blog will talk about embedded programs, not Windows, not Unix, but embedded programs. Some written exclusively in assembly, some in C. Most with no threads or other OS assistance.

Definitions: Software Archaeology - The investigation, research, documentation, and rewriting to gain meaningful understanding of long ago abandoned or neglected software programs.

What causes a program to be abandoned or neglected? Why is the archaeology required in the first place?

The programs I have worked with were written in the early '90s. Software standard practices are better than back then. I will say that many projects are better than back then, but there are many that are still built the same way it was done 20 years ago.

Software consultant, Joe, talking to friend consultant John.

"Joe - How's the new assignment going?" ask John. "Oh, they're writing legacy code" replied Joe.

When software is written it combines 1) the author's domain knowledge, and 2) the author's understanding of the underlying hardware.

The software is constrained by how well the underlying hardware can accomplish the task to be completed. The software is also constrained by the author's knowledge of the domain problem that is the source of information for the task. The author then brings their personality, experience, drive, and insight to the writing of software.

Software is the art and science of translating human goals into a language where a computer can perform the task expounded in the goal.

What do I find when I read code from another era? I find the remnants of enough of the domain and language knowledge to do the job, but no more.

I took a computer languages course in 1976. I was introduced to Algol, PL/1, Snobol, and APL. I do not use any of these languages today. I don't know who does. I learned and used FORTRAN in other courses, which is still widely used in numerical computing applications. C was just starting to be used in research labs.

If I had to resurrect a program from that era, I would have to learn, to a certain extent, the actual computer language, its syntax and nuance, to understand how the program functioned.

Sometimes, the need for the source code is not necessary. Sometimes all that is required it a complete definition of the inputs and outputs of the program. This is probably a simple program, but if you know that a certain list of numbers goes into a program and a set of operations is performed on that list and a new new set of numbers is created, then any programming language that handles the inputs and outputs could be used to satisfy the task of converting the input to the output.

The constraint on recovering an old program is that the inputs and outputs are still in place. They cannot be changed. What is missing is the exact definition of the inputs and outputs. The program knows what those inputs and outputs are. But the complete definition is in the code.

Unfortunately, the program can't expound on its nuances or give background on what the author was thinking. Comments help, when they are present.

The techniques of re-factoring are those that will provide the most insight with the best chance of documenting the inputs and outputs.

The other difficulty with working with old programs are the tools. With each generation of processors, comes a new generation of tools.

In the '90s the method by which one debugged an embedded program was very primitive or very sophisticated, but not much in between. The primitive method was to output RS-232 messages to display the current state of the code. Each output would reveal the changing state. Analysis would then determine what might be wrong. The very sophisticated, and thus very expensive method, was to use an In-Circuit Emulator or ICE.

Memory was expensive in the '90s. Embedded processors did not have cache. Programs ran from Read Only Memory, which may have been PROMs, EPROMs or Flash. The processor would have break point capability, but only if the memory location could be changed to an 'illegal instruction' to cause a jump to the interrupt handler that would provide the debugging support. This only worked if the program was running in RAM. Inserting an illegal instruction into ROM is impossible. This is the same mechanism used today for software breakpoints. Hardware breakpoints were nowhere to be seen.

The ICE provided a way for a host processor, a PC, to have RAM memory substitute for ROM memory as well as take over the clock operations of the processor, allowing the user to watch the processor step through each instruction in as much detail as desired.

Breakpoints are essential.

RS-232 would disturb the timing of the program and use up precious memory. The ICE was an emulator and thus provided the debugging functions without the rewriting of code and using any additional memory.

If the program was neglected, then the tools have been neglected. The ICE unit may no longer power  on, if it can be found at all.

The history of how the author came to writing the code is lost. The author learned, most likely by trial and error, the nuances of the language and the hardware. This history is not documented.

All in all a puzzle.

That's another good definition of software archaeology, the study of puzzles created by time and neglect.

Tuesday, September 8, 2015

Number of Processors, Linux, and Make

[Much of this post is from here,]

For those who use Windows, there is an environment variable that comes pre-loaded, NUMBER_OF_PROCESSORS.

This variable contains the number of processors that a program might want to know to allow for parallel operations.

But in this day and age, the number of actual processors and the number reported may differ by a factor of two. The technology that allows this is called hyperthreading. Hyperthreading has been around for more than ten years.

A CPU chip may have two physical cores, but with hyperthreading, the operating system is presented with four processors.

Under Windows,  NUMBER_OF_PROCESSORS is the hyperthreading number.

Under Linux, we can run a set of commands and get both.

# Get the number of actual processors, not hyperthreaded processors
NUMBEROFPROCESSORS=`cat /proc/cpuinfo | grep 'core id' | wc -l`
echo "Number of Processors="$NUMBEROFPROCESSORS

# Get the number of hyperthreaded processors
NUMBEROFHTTPROCESSORS=`cat /proc/cpuinfo | grep processor | wc -l`
echo "Number of Hyperthreaded Processors="$(($NUMBEROFHTTPROCESSORS-$NUMBEROFPROCESSORS))

The environment variable NUMBEROFPROCESSORS is created by the output of the file /proc/cpuinfo being piped to grep, which looks for the string processor or core id. The result is then piped to wc which counts the number of lines grep found.

My results are as follows. I'm running on an Intel(c) Core(tm)2 T9400. It does not support hyperthreading.

Number of Processors=2
Number of Hyperthreaded Processors=0

We can use this information with the -j argument for make.

# Get the number of actual processors, not hyperthreaded processors
NUMBEROFPROCESSORS=`cat /proc/cpuinfo | grep processor | wc -l`

Now make will use the number of processors (with hyperthreading). We don't have to change the make script each time we get more cores on our VM or laptop.

It is mentioned (can't find the URL) that -j has no effect when running under MS-DOS. Don't know if this has changed. The version of Windows is not mentioned. nmake, supplied with Visual Studio, is said to be available for multi-core (parallel) operation. StackOverflow discussion here.

Using -j, in general, assumes that the build is not a highly dependent build. Lots of leaves in the build tree.

More details on cpuinfo can be found here.

Nice discussion on the why's and wherefore's of processors, physical ids, and cores here.

Busy - Sluggish

Now that you have enabled make to use all the available processors, be prepared for your system to get sluggish.

On a build machine you want to build fast. On a development machine you want to do something else, but you just told make to use all the available CPUs. Ctrl-C might not even be effective, for a while.

Your mileage may vary. Forewarned is forearmed.

Number of Processors + 1

There are many discussions that state the value for the make argument -j should be the number of processors +1, even +2. The idea is that make (and compiling) is an I/O intensive activity. While waiting for one compile to finish, another can be prepared.

At this URL, Agostino Sarubbo, has shown that one should use the number of processors and no more.

The reason for this would be a subject of a separate blog. Exercise for the reader. :)


All the in-line links listed in one place.