FTDI-Gate: The WRONG way to fight counterfeits

NOTE: This article was written in 2016 for an internal group. I’ve decided to publish here for posterity
What or Who is FTDI?

FTDI is a major developer of USB communication chips and drivers. Their most common product is a line of serial to USB chips and drivers. If you have a USB to serial converter or a black box that connects older equipment to a USB port, chances are there is an FTDI chip at the heart of it.
FTDI also provides drivers for their chips. A driver is a piece of code that runs on your computer that tells Windows how to talk to the device.
The FTDI line of USB chips are incredibly popular and common in many electronic widgets. They are found in test equipment, DIY products (Arduinos, BeagleBone’s, Raspberry Pi etc) as well as many consumer electronics. Anything that speaks USB and Serial probably has an FTDI chip somewhere.

The Problem When You’re The Best:

FTDI chips are so reliable and popular that the market is awash with counterfeit FTDI chips. These fakes look just like genuine FTDI chips and can be impossible to distinguish without drastic testing. Often the only way a consumer can tell is by replacing a buggy piece of hardware with a different vendor. The real problem with the counterfeit chip business is that the end user or even a mid level producer often has no means of knowing if the chips they are getting are genuine or not. The supply chain is so heavily saturated with counterfeits, even distributors have a difficult time guaranteeing genuine parts.

FTDI’s Answer to Counterfeiting – FTDI-Gate Part 1:

In late 2014, rumors and blog posts started appearing describing problems with an updated FTDI driver being pushed out through Windows Update. Although not documented by FTDI, it was soon discovered that this new driver would modify the counterfeit chip in a specific way, rendering it useless. In the community this is known as “bricking” the device. The driver would alter a parameter in the chip that would stop it from functioning with any other computer. This was FTDI’s answer to the counterfeit chips, to disable them without the end users knowledge or consent.
As one could imagine, the online community erupted against FTDI. This action was seen as duplicitous, manipulative and possibly illegal. An analogy to this would be Honda remotely destroying your car because they discovered a counterfeit part in the engine. Punishing the end user for market problem far upstream was a very heavy handed action by FTDI who were forced eventually to rescind the damaging driver update, although bricked devices were not repaired.

FTDI Strikes Again – FTDI-Gate Part 2:

Fast forward to February 2016. Stories began to pop up about strange, random behavior of serial to USB adapters. With the stink of FTDI-Gate still fresh in peoples memories, the builder community began to suspect FTDI of shenanigans once again.
Like a rerun of a bad sitcom, it was soon determined that FTDI had once again modified it’s drivers, pushing a new poison pill out through the Windows Update service. This time, if the driver detected a counterfeit chip, the driver would inject text into the data stream. Specifically the driver would inject “NON GENUINE DEVICE FOUND!” into the serial and USB data streams.
This is different than bricking the fake chip, and in some ways worse. Imagine for a minute you have an old chiller that communicates to a BMS system via a serial to USB converter, which is not such an uncommon occurrence. The chiller receives commands and sends data via this serial to USB converter. Suppose again that this serial to USB converter was purchased from a major retailer like Best Buy, and was manufactured in China. The changes of having a counterfeit FTDI chip is probably 60/40. How would the chiller respond if it suddenly received “NON GENUINE DEVICE FOUND!” from the communication line? How would the BMS system respond to receiving this string from the Chiller? There really is no way of knowing.
Injecting invalid data into a production system is akin to sabotage, resulting in unpredictable results. Bricking the device at least leaves one in a known state.
This firmware update has not yet been revoked.

So Now What?

I wanted to bring this story up because it’s a good illustration of the types of struggles with electronic systems and why they suddenly stop working. In this case, there really is nothing the end user can do to ensure they are getting genuine FTDI parts other than being particular in where they buy products. Also as a warning to everyone who uses USB to Serial converters. If you happen to one day run into problems with the converter, you may have been affected by the FTDI-Gate.

Organizing Memories

So many memories we’ve forgotten

When was the last time you had pictures printed? How many pictures do you have on your phone or tablet? Do you even remember what pictures you have anymore or where they’re from?

My wife’s grandmother passed away 2 years ago. Her grandfather, John, and his wife were prolific picture takers and frequent vacationers. Over their last 5 years or so they had amassed several thousand pictures on various devices. Most of these were on two iPad’s, the balance being spread across a few CDs and usb flash drives.

So the problem was presented by the family to me. We have all these pictures, we don’t know what is a duplicate of what, we don’t know what’s on these drives or the ipads but we need it all organized in a single folder on a new laptop. The intent was for John to be able to view and curate these pictures on a laptop (that I would purchase and set ip), which was something he was struggling with on the ipad.

In a very rare act of taking the high road, I’ll skip my usual hate rant for Apple products, their apparent hatred for their user base, and their obvious goal to frustrate those of use who serve as the family tech support.  Suffice to say that I managed to invoke the right magic, spoke the correct incantations and beckoned the  appropriate powers to get all the pictures off of both ipads and safely onto my machine. We’ll skip the pain and move on.

All told, I now had around 2600 pictures to work with and I had two objectives in mind.

  1. Eliminate any duplicates in this collection of pictures
  2. Sort this collection of pictures in to a folder structure for the years and months the pictures were taken.

Finding Duplicates

This was an interesting problem to solve. I am very sure this wasn’t the most efficient way to solve the issue but it was rather interesting to me. My approach was to read through the various folders for the picture files. For each file, I load the filename, extension and path into a database. I also take an md5 hash of the picture as well as the date the picture was taken as reported in the exif data. These additional data points are also loaded into the database.

For those unaware, an md5 hash is a unique fingerprint of the data within the file, which for my purposes is enough to uniquely identify the file based on its content and not on its file name. Also, exif data is data saved inside a picture file the identifies several properties of the picture such as the camera it was taken on, the camera settings and the date the picture was taken.

From here, weeding out the duplicate pictures was a trivial SQL query to select the unique md5 hashes. Now I thought that was pretty neat. See the code for this first stage below. You can see that I am using exiftool to pull out the exif data. Its a great tool written by Phil Harvey.  Also below is the end result of this script

if [ $1 -z ];
then
echo "Need to include the root directory containing pictures"
exit
fi
if [ $1 = '-h' ];
then
echo "   [-d]:to create duplicate database "
exit
fi</code>

search_dir=$1
#echo $search_dir
first=0
find $search_dir -iname '*' -print0 | while IFS= read -r -d '' entry
do
if [ $first -gt 0 ];
then
filename=$(basename "$entry")
extension="${filename##*.}"
path=$(dirname "$entry")
md5hashString=$(md5sum "$entry")
md5hash=($md5hashString)

imageDate="$(exiftool "$entry" |
grep -m 1 "Date/Time Original" |
sed "s|Date/Time Original : ||g" |
sed "s|:|-|g")"

if [ -z "$imageDate" ];
then imageDate="$(exiftool "$entry" |
grep -m 1 "File Modification Date/Time : " |
sed "s|File Modification Date/Time : ||g" |
sed "s|:|-|g")";
fi;
if [ -z "$imageDate" ];
then
imageDate="1980-01-01"
else
imageDate=($imageDate)
fi

sql="INSERT INTO Pictures (Path,File,Extension,Hash,Date) VALUES ('$path', '$filename', '$extension','${md5hash[0]}','$imageDate')"
mysql -D GGPics -e "$sql"
fi
first=1
done

Picture hashes

Organizing the Final Product

Once I had everything in the database, I made a new table of distinct hashes, essentially weeding out the duplicate files. Next was the organizational step which was rather trivial. For this, I wrote a small python script (again probably not as efficient as it could be) to accomplish the organization. The script reads through the unique table, determines if a folder exists already for the year the picture was take, and the month the picture was taken. It creates the folders as needed and then copies the image into the new structure. Code and results below

oldPath = result[1]
oldFilename = result[2]
oldDate = result[3].strftime(‘%Y-%m-%d’)
oldFile = open(oldPath + "/" + oldFilename, "r")

#see if year folder exists
if not os.path.exists(newLocation + "/" +oldDate.split("-")[0]):
os.makedirs(newLocation + "/" + oldDate.split("-")[0])
month = calendar.month_name[int(oldDate.split("-")[1])]
#see if month file exists
if not os.path.exists(newLocation + "/" + oldDate.split("-")[0] + "/" + month):
os.makedirs(newLocation + "/" + oldDate.split("-")[0] + "/" + month)
if((oldDate.split("-")[0] == "1980") or (oldDate.split("-")[0] == "1900")):
newPath = newLocation + "/" + "UnknownDate" + "/"
filecount = len(os.listdir(newPath))
newFile = open(newPath + str(filecount+1) + ".jpg","w")
else:
newPath = newLocation + "/" + oldDate.split("-")[0] + "/" + month + "/"
if os.path.isfile(newPath + oldDate + ".jpg"):
filecount = len(os.listdir(newPath))
newFile = open(newPath + oldDate + "_" + str(filecount+1) + ".jpg","w")
else:
newFile = open(newPath + oldDate + ".jpg","w")
newFile.write(oldFile.read())
newFile.close()
oldFile.close()
lastID = lastID+1
print newFile.name
print str(lastID) + " of " + str(totalFiles) + " done"
curs.execute("UPDATE DistHash SET Processed=1 WHERE ID=" + str(result[0]))
conn.commit()

At the end I end up with a structure that looks something like this. I don’t have the actual pictures anymore so its showing 0 files but you get the idea.

Picture Structure

All in all, this was a pretty fun project I intend to work on a little more and use for my own organization. My wife is a prolific picture taker and over the years I have multiple backups. So having this type of tool to quickly organize and weed out duplicates is pretty useful.

More importantly, this tool gave me the ability to give something important to John. It let me give him back his memories. So often we leave our memories locked up in our devices. I was pleased to be able to give John access to the pictures he hadn’t seen in a long long time and the memories that are attached to them. It reminded me of how precious and fragile our memories are.

“Daddy, lets build electronics!” – How a book changed my life

This is less an interesting technical blog post and more just a retrospective on my past and what it has given me today.

As long as I can remember I’ve loved electronics, since the first time I took apart a tv remote when I was 12 (much to my fathers unhappiness). Looking at the green epoxy coated circuit board, it made me think I was looking down at a green landscape, resistors and capacitors dotting the scenery like towns and villages. Then I was just an observer discovering new worlds. Eventually in highschool I began learn electronic components and design, to which I am always grateful to Mr.Langil my electronics teacher. Now I felt like a creator of landscapes, not just an observer. I was hungry for knowledge and without the easy access the internet provides us, I turned to books.

Sometime in 2000 I picked up a copy of “Engineer’s Mini Notebook: Timer, OpAmp & Optoelectronics Circuits & Projects” from Radio Shack.  This was the first I had ever seen of this book series  or its incredible author, Forrest Mims. Initially I had been interested in it because I had worked with 555 timers in the past and Op Amps were like a new magical power waiting for me. But after looking at this small book I was captivated by its hand drawn diagrams and notes. There was a romantic, real quality that I couldn’t resist. This wasn’t a text book, this was like a hand written note and treasure map directed to me. I can’t say how many hours I spent with this book or how many of these circuits I built, but this book took me from a dilettante to someone with a real passion to build.

Fast forward through highschool, two rounds through university and a decade and a half in various careers. Through the wonderfully deterministic world of digital electronics, the seductive magic of analog signals, and finally programming and embedded systems where I could finally communicate with something.  Now I spend most of my time writing code and less time building hardware but now and then will still pull out my old kits and put something together.

My 5 year old daughter spent a week this summer at a STEM camp. During that week she built a simple flasher circuit out of conductive play-dough, a battery and an LED. She loved this project and was so excited to show me, knowing her daddy is an engineer. She told me all about how it worked, how the electricity moved through the metal and the dough to make the light work. She wanted to build more and decided that we would build electronics together.

My home office is littered with containers of components, half finished projects and books. I started scrounging around for bits and pieces to put together while my daughter picked through my bookshelves. It may have been the colorful cover that drew her attention but for whatever reason she picked up this same “Engineer’s Mini Notebook” that had caught my attention so long ago. She asked if I had drawn the diagrams and pointed out to me the LED and resistor symbols she had learned.

I would be lying if I said I wasn’t a little emotional seeing her get so excited over this same book that had started me on my journey.

I picked a simple project that would appeal to her and the next day we were down at the local electronics shop to pick up the bits I was missing. An hour of breadboarding later, my daughter learning to use a multimeter to measure capacitors, and we had our first project. She loved it and wouldn’t stop playing with it. Breadboards are temporary and this was a memory I wanted to last. So the next day while she was at school I soldered it up on a prototype board. And here it is, our first Father Daughter project.

First Project

This is the Toy Organ project from page 23 of Forrest Mims’ “Timers, OpAmps & Optoelectronics” entry from the Engineer’s Mini Notebook series. And it still sits on the window sill of my daughters room. Every now and then when she’s in her room I can hear the tones from the circuit.

I can’t express how grateful I am to Mr.Mims and his book series. It played an important role in my life and is now helping me introduce my daughter to the exciting adventure of world building. We’ve already picked the next project to build, maybe one that doesn’t make so much noise this time. I look forward to many weekends spend with this book on one side, and my daughter on the other.

Geographic Lookup From IP Address

Grepping through Log Files

Managing a few small web sites and some web applications, I’m constantly finding myself grepping through log files looking at access activity  and suspicious entries. I’m always interested in looking up the geographic location of where access requests are coming from. Is the traffic local or am I getting hit from Russian or Chinese IP addresses? There are a lot of online tools to get geographic information from an IP address, such as here and here, buts its time consuming when you want to look up lots of addresses. So I wanted a script to do more of a bulk lookup.

IP lookup API’s for scripting goodness

While looking around the internet for a good, free API for geographic lookup, I found ipstack, a free IP Geolocation API service. Signup is required in order to get an API key, but the free version should give you most of the functionality needed, with 10,000 requests per month. The API is pretty easy to use also, and a simple curl request can do the trick. The API returns the results by default in JSON format.

In this example I’m making a curl request to the ipstack api, requesting the geographic location for Cloudflair’s 1.1.1.1 DNS server. The response comes back in JSON so I’m piping the output to jq, which is a really handy JSON parser for bash.

curl http://api.ipstack.com/1.1.1.1?access_key=xxxxxxx | jq

The ipstack API has different options to only request some of the data fields and not others. I’ll leave it to the reader to look them up if interested.

Parsing Log Files for feedstock

So this was a really good start, but I want to script multiple requests to the API and store the results in an sql database for easy lookup. I’m a big fan of using databases for information storage and usually opt for using sql even if its a bit overkill.

Apache is generally pretty good at keeping log files. The default apache config keeps aceess logs in /var/log/apache2/access.log , where errors are kept in /var/log/apache2/error.log. This may be different on your system, so RTFM. These files are nothing if not dense but they are reliably formatted. Awk is your friend here. You can use awk to pull out the log file IP addresses. From there, the rest is easy peasy.

sudo awk '{ print $1 } ' /var/log/apache2/access.log | sort | uniq

I’m piping the results of awk through sort an uniq, which sort the results numerically and strip out duplicate results. If you have multiple log files you could always cat them together before sending them through awk.

Now the magic happens

Now we want to run through each IP address, make the request from ipstack and store the geographic results. Its possible to do this purely with bash,but my bash game isn’t quite that strong. I chose to write a small Go program to do the job.

package main

import (
"database/sql"
"encoding/json"
"fmt"
_ "github.com/go-sql-driver/mysql"
"io/ioutil"
"net/http"
"os"
)

type Result struct {
IP string
Continent_name string
Country_name string
Region_name string
City string
}

func main() {
var myResponse Result
args := os.Args[1:]

if len(args) != 1 {
panic("Need an arguemnt")
}
target := args[0]
fmt.Printf("Using target %s", target)
url := "http://api.ipstack.com/" + target + "?access_key=xxxxxx&fields=ip,continent_name,country_name,region_name,city"
resp, err := http.Get(url)
if err != nil {
panic(err)
}

defer resp.Body.Close()
contents, err := ioutil.ReadAll(resp.Body)
err = json.Unmarshal(contents, &myResponse)
if err != nil {
panic(err)
}

//connect to database
db, err := sql.Open("mysql", "ipLookup:@/IPTrack")
if err != nil {
panic(err)
}

stmt, err := db.Prepare("INSERT INTO Results (IP,Continent, Country,Region,City) VALUES (?,?,?,?,?)")
if err != nil {
panic(err)
}

_, err = stmt.Exec(myResponse.IP, myResponse.Continent_name, myResponse.Country_name, myResponse.Region_name, myResponse.City)
if err != nil {
panic(err)
}
db.Close()
}

Remember, my standard Code Caveat applies.

You can pipe the output of the awk command above to this program, which will load the ipstack api lookup results to sql. What you’re left with is a handy database full of geographic locations of your visitors. Pretty neat.

This makes it really clear this particular server is getting a lot of attention from China. This is good information to now make a decision as to whether I want to impose a geographic firewall rule or not. I’ll leave that up to the reader to decide.