 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
| HOME · MOVIES · HIKING · PHOTOS | SITE MAP | ||
One will not find information or instructions for how to circumvent transit fares of any sort on this site; if that's what you're looking for then you're in the wrong place. Neither the author nor this site advocates or condones ripping off transit.
Please note that the information presented here likely has errors, as it's based on educated guesses and conjecture about data obtained from tickets that were almost all found on the streets.
|
|
|
|
|
|
|
|
|
|
Compass is TransLink's contactless NFC fare payment system. It was deployed after some delays, in stages, during the second half of 2015 with the final release occurring on Nov 2, 2015 when Compass Cards were made available to the general public via FareDealers. The author is a fan of NFC technology and was eagerly awaiting the deployment for the practical reason of reducing the number of coins in his pocket and the geeky reason of wanting to play with NFC cards and tickets to explore the product. Compass would also see TransLink keeping up with successful NFC-based transit fare payment systems around the world. However, the author noticed an obvious product omission: There is no smartphone app to read the card or ticket in order to display the fare product's expiry times or the stored value balance.
Neither the cards nor the tickets can have a fare-expiry date printed on them; cards for obvious reasons, and the tickets because the expiry is dependent on the time of first-tap. This lack of a visible expiry date, despite the technical reasons, is a step backwards in usability from the magstripe ticket system that Compass replaced. A smartphone app to display the expiry times would be a technologically appropriate remedy that would be a similarly appropriate step forwards in usability, except for those without smartphones (but there was also a time when many people couldn't read, which didn't stop books from being published).
This research project was started in order to discover if it would be possible for a third party to write such an app, given the information that is available to the public (i.e. the data in the tickets). It was quickly discovered that Compass Cards require a key even for read-only access, thus Compass Cards are currently off the table for any sort of analysis. Attention was turned towards Compass Tickets, which are unencrypted and readable by any NFC-capable device.
Over the span of a few months, more than 100 Compass Tickets were collected, almost entirely as transit litter. These were scanned with a smartphone, the data was saved, then analyzed for obvious data structures and values. Many tickets were taken to a TVM in order to run a View Balance request to determine what the ticket data might represent.
The Compass Ticket uses the MIFARE® Ultralight® chip, although a few examples have been found which use the Ultralight EV1. The Ultralight chip has no means to prevent data from being read but a secure product doesn't need that. The Compass data is unencrypted and appears to be protected from alteration by a Message Authentication Code (MAC) on each data record.
The basic Ultralight product has apparently been end-of-lifed and replaced by the Ultralight EV1. As of Summer 2017 it appears that supply channels may finally have run out of the Ultralight chips, so it is expected that very soon all Compass Tickets will use the Ultralight EV1 chip.
The Compass Card uses the MIFARE DESFire® EV1 product. Associated with the Compass application on the card is a read/write key and a read-only key but neither of them is publicly known, neither is the master key for the card. As of this writing, the DESFire EV1 does not have any publicly-known security vulnerabilities that might allow the read-only key to be obtained. Thus no analysis of the Compass Card is possible at this time.
A Compass Ticket consists of a layer of card stock on either side of a plastic inlay onto which an aluminum coil has been etched and the silicon die bonded. These three layers are cemented together with an adhesive. The reverse has the Batch ID Number and the Compass Number printed near the bottom edge.
To disassemble a ticket without damage one can use a petroleum solvent. This does not appear to attack the ink used to print the Batch ID Number and Compass Number. A particularly successful technique is to stand the ticket on its end in a small pool of solvent and allow it to wick up into the ticket, softening the adhesive. After 30 - 60 minutes the ticket is easy to peel apart without damaging the card stock and can be cleaned of any residual adhesive. The die remains bonded to the inlay and should still be functional. Some manufacturers may use cements which are affected by solvents more slowly, so some testing may be required.
Each ticket has a six letter code printed on the reverse at the bottom left corner. The code has two letters and four digits (in the form of two numbers). The two letters must be the manufacturer ID because they correlate with names that are visible on the inlay of many tickets when a light is shined through them. That, plus some 'net searches, indicate some likely suspects. Here's a speculative list.
At first it was believed that the numbers were two separate codes but with enough tickets collected it became apparent that they were more likely to be a two-digit year and a week number. A number of points were considered before this became apparent.
At this time it's not known what this point-in-time is marking. It doesn't seem to be a manufacturing date because there is evidence of multiple production runs that use the same code.
There are four top-level structures on each Compass Ticket: Manufacturer data including the Unique Identifier, a Compass Product Record, and two Compass Transaction Records.
Here's the data from a real Compass Ticket as an example, organized into the 16-byte record size used by Compass.
04F738437A624380DB48000000000000 ← Manufacturer Data 0A04003D200182000000002E0000875E ← Product Record A04502FF0100040001000000CA46C62C ← Transaction Record 2648021A020000000193170577ABBBA6 ← Transaction Record
The Ultralight EV1 chip has 16 bytes more than the original Ultralight. They are currently unused by Compass.
000000FF000500000000000000000000 ← EV1 Configuration Pages
The main item of interest in the Manufacturer Data is the 7-byte Unique Identifier (UID). This is the source of the card or ticket's Compass Number.
To derive the Compass Number from the UID:
For the above sample ticket, the UID to Compass Number transformation steps are as follow:
04F738437A624380DB48000000000000 ← Manufacturer Data 04F738 7A624380 ← UID UID 04f7387a624380 → 0xf7387a624380 → 271821943489408 → 0001 271821943489408 9 → Compass No: 0001 2718 2194 3489 4089
The Product Record does not appear to change during ticket use. The addition of an Add-Fare is handled by the carry-forward of the stored value balance in the transaction records. TransLink's online documentation and TVM screens calling the addition of Add-Fare a "ticket upgrade" is somewhat confusing as, to the author, the word upgrade implies that the product will be changed to a higher-valued one.
Known offsets:
For the above ticket, the values break down as:
0A04003D200182000000002E0000875E ← Product Record
0A ← Unknown; always 0x0a in Product Record? Data structure version number?
04 ← Ticket Type: Adult
00 ← Unknown; always 0x00 in Product Record?
3D20 ← Date Code: 0x203d == 2016-01-29
01 ← Unknown (possibly the number of service days?)
82 ← Product Code: "1 Zone"
00000000 ← Unknown; always zero in Product Record?
2E ← Mystery Byte: Unknown
0000 ← Unknown, always zero before June 2017, then Machine Code
875E ← MAC
The Date Code represents 2016-01-29 (see Date Representation, below). This is merely a base date which will be adjusted for each transaction by the Date Offset found in the Transaction Record.
In June 2017, possibly on 2017-06-14, the two zero bytes of the Product Record word at offset 0x0c began to store the Machine Code from the purchase Transaction Record; thus, they store the number of the TVM that vended the ticket. Why this change was made is unknown but it is logical; unfortunately, the change didn't go far enough!
So, what's missing? This change results in the Product Record having the same problem as the purchase Transaction Record; namely, that the word corresponding to the Location Code is zero just like it is in the purchase Transaction Record. This is a problem because the Machine Code is subject to 16-bit truncation at stations with an ID number greater than 65, and without a non-zero Location Code it is impossible to be certain that one has correctly adjusted for this truncation.
Known offsets:
For the above ticket, the values of the two Transaction Records break down as:
A04502FF0100040001000000CA46C62C ← Transaction Record, purchase
A045 ← Time (0x45a0: 11 bits 0x22d → 09:17; 5 bits 0x00 → purchase record)
02 ← Date Offset (2016-01-29 → 2016-01-27 for this transaction)
FF ← Unknown; always 0xff for purchase?
010004 ← (0x040001: 6 bits unknown; 11 bits Stored Value; 7 bits Sequence Number)
Unknown = 0x01, Stored Value = $0.00, Sequence Number = 1
00 ← Minutes since first tap-in; always 0x00 for a purchase?
01 ← Expires Offset (expires before 2016-01-28 service day)
0000 ← Location Code; always 0x0000 for a purchase?
00 ← Line Code; always 0x00 for a purchase?
CA46 ← Machine Code (0x46ca == 18122 → Gateway Stn TVM)
C62C ← MAC
2648021A020000000193170577ABBBA6 ← Transaction Record, tap-out
2648 ← Time (0x4826: 11 bits 0x241 → 09:37; 5 bits 0x06 → tap-out record)
02 ← Date Offset (2016-01-29 → 2016-01-27 for this transaction)
1A ← Zone Code(?) (changes as trip progresses)
020000 ← (0x000002; 6 bits unknown; 11 bits Stored Value; 7 bits Sequence Number)
Unknown = 0x00, Stored Value = $0.00, Sequence Number = 2
00 ← Minutes since first tap-in (wraps after 0xff?)
01 ← Expires Offset (expires before 2016-01-28 service day)
9317 ← Location Code (0x1793 may be a bus stop but no lookup data is available)
05 ← Line Code: Bus
77AB ← Machine Code (0xab77 == 43895 but no lookup data is available)
BBA6 ← MAC
The date system is straightforward once one knows how the bits are layed out. There are three bytes that are known to be used for the determination of a transaction date. The author speculates that a fourth byte is used by multi-day DayPasses to determine the number of service days remaining (a MultiPass product is listed in the Transit Tariff).
The Product Record contains two bytes, at offset 0x03 and 0x04, which determine a base date. These bytes are a little-endian 16-bit value (with an odd address alignment, and crossing a 32-bit boundary, both sure to offend CPU purists!); the author calls this the Date Code. The 16 bits break down into three fields: Seven bits for a count of the years since 2000, four bits for the month number, and five bits for the day number. It's not known if the years bits form a signed number.
The years begin at zero but the month and day begin at one. There will be discontinuities in the Date Code because of the 4-bit month and 5-bit day number fields. Robust code needs to check for undefined dates (always sanity check input data!).
Each Transaction Record has an unsigned 1-byte Date Offset (at offset 0x02) which is the number of days to offset (by subtraction) from the date stored in the Date Code in order to obtain the date of the Transaction Record.
Here's an example using data from the above ticket.
Product Record: 0A04003D200182000000002E0000875E
Date Code 0x203d → 0010000 0001 11101
YYYYYYY MMMM DDDDD
Year = 16 + 2000
Month = 1
Day = 29
Transaction Record: A04502FF0100040001000000CA46C62C
Date Offset 0x02 → Transaction date is 2016-01-27.
Observations show that the Date Offset in the purchase transaction record of TVM-issued tickets is always 0x02; this, combined with the Date Code being two days ahead does indeed record the current date. Further observations show that for a ticket purchased before midnight, any transactions that occur after midnight will have a Date Offset of 0x01; thus, Date Offsets follow the wall clock calendar, not the Compass business day that ends at 04:00.
Some examples taken from real tickets (except the undefined dates which are deliberately used as examples of bad Date Codes (which should not be encountered in actual practice (if one is writing code then it needs to handle them))).
| Date Code | YMD | Date Offset | Date |
| 0x1f9d | 0001111 1100 11101 | 0x02 | 2015-12-27 |
| 0x2022 | 0010000 0001 00010 | 0x02 | 2015-12-31 |
| 0x205d | 0010000 0010 11101 | 0x02 | 2016-02-27 |
| 0x205e | 0010000 0010 11110 | — | undefined |
| 0x205f | 0010000 0010 11111 | — | undefined |
| 0x2060 | 0010000 0011 00000 | — | undefined |
| 0x2061 | 0010000 0011 00001 | 0x02 | 2016-02-28 |
| 0x2062 | 0010000 0011 00010 | 0x02 | 2016-02-29 |
| 0x2063 | 0010000 0011 00011 | 0x02 | 2016-03-01 |
| 0x215b | 0010000 1010 11011 | 0xfe | 2016-02-16 |
Alert readers may have spotted an ambiguity – here's how it's resolved: The Date Code must be for a valid calendar date; it is not correct for the Date Code to identify an invalid calendar date regardless of whether or not the Date Offset would adjust it to a valid calendar date.
The first two bytes of a Transaction Record are a little-endian word consisting of two bit-fields: The most significant eleven bits are the time of day (as minutes since midnight) and the five least signficant are the Transaction Record Type. There are no seconds.
Here's an example using the same ticket data as above, for which the Date Code and Date Offset shows that the transaction is for 2016-01-27, but at what time?
Product Record: 0A04003D200182000000002E0000875E
Transaction Record: 2648021A020000000193170577ABBBA6
Time: 0x4826 → 01001000001 00110
Minutes Type
→ 0x241 minutes since midnight; Transaction Type = 0x06
→ Transaction time is 09:37; Transaction Type = Tap-Out
That's it!
It was initially believed that the entire 16 bits was for the time of day, which led to some interesting theories but it seems reasonable that it was never that way at all. However, it is very interesting to pretend that the entire 16-bit word is for time, and then follow a line of thought to see where it leads – so what follows is a pleasant academic diversion.
Any technical person knows that the problem with a 16-bit word for a time of day is that there are more seconds in a day than can be stored in 16 bits. The naïve approach is to divide the seconds by two and store that number; problem solved, except that divide-by-two has some practical problems, and there is a better representation, as described below and named 60/32 Time for convenience.
An analysis of transactions, using a TVM, suggested that instead of dividing by two, the designer apparently deemed the unit of time to be the minute and then stored the count of 1/32 minutes since midnight. The clock tick for this representation has a period of 60/32 (1.875) seconds instead of the exactly 2 second period that the divide-by-two approach has.
This was puzzling but the author assumed that the designer wasn't just being silly, so he researched time systems. Of particular relevance was the time system used historically for railroad schedules; is it a coincidence that SkyTrain is a rail system? It was found that fractional minutes such as ¼, ½, and ¾ were printed in old schedules. Some math shows that the seconds-divided-by-two representation cannot exactly represent ¼ or ¾ minutes but that the 1/32 minute representation can; in fact, it's exact for any multiple of 1/32 minute. For railroad schedules this implies that this time representation could be used for a precision of ⅛ minutes or better, if needed, but it's not known to the author if any railroads actually schedule to a 7.5-second(!) precision.
If 17 bits were available as a natural word size then the time could be stored with to-the-second precision but there wouldn't be any opportunity to study this very interesting history.
Factoids about 60/32 time:
Let's look at the above example but instead with this time system.
Product Record: 0A04003D200182000000002E0000875E
Transaction Record: 2648021A020000000193170577ABBBA6
Time as 60/32 count: 0x4826 * 1.875 = 34631.25 seconds since midnight
→ Transaction time is 09:37:11.25
Time as fixed-point float: 0x4826 = 01001000001.00110 minutes
→ 0x241 3/16 minutes since midnight
→ Transaction time is 09:37:11.25
If seconds are not desired:
→ 0x241 minutes since midnight; other = 0x06
→ Transaction time is 09:37; other = 0x06
Wasn't that interesting? As mentioned below, it was eventually realized that the "seconds" of transaction times were limited to particular values. Initially, this suggested that the seconds bits were being overloaded with another purpose but it was deemed to be more likely than not that there never had been any seconds stored and the 16-bits actually held two unrelated fields.
The Sequence Number and Stored Value balance are unrelated but both numbers are packed into the same 24 bit little-endian word: 6 bits of unknown purpose (most significant), 11 bits for Stored Value, and 7 bits for the Sequence Number (least significant).
The sequence number is initially zero for an unused record and then monotonically increments by one for each new record. It wraps after 0x7f but nothing exciting happens when it does. If one maintains a history for a very busy ticket then it's obvious that the sequence number in that set of history records cannot be depended upon to be unique.
A Compass Ticket that has been upgraded with Add-Fare for travel into additional zones has the Add-Fare amount added to the Stored Value, as pennies. With 11 bits available, at most $20.47 can be stored, which seems to be a reasonable maximum for a disposable ticket product. Note that the TVM's don't allow money to be added to tickets explicitly as Stored Value, only as Add-Fares for extra zones (plus the $5.00 if you're upgrading a free ticket on Sea Island) but a TVM View Balance will display the amount as Stored Value.
Add-Fare ticket upgrades are not yet fully understood. A test was performed in Zone 1 to upgrade a one-zone ticket for a second zone. This created a transaction record with a type code of 0x04. A second upgrade for the third zone was performed, which created a transaction record with a type code of 0x00. Very strange; why would the same operation have a different transaction type, one which coincides with what is assumed to be the initial product load?
It's not yet known how a ticket knows which zones have been paid for. Perhaps the above has something to do with this but the data is not carried forward with future transactions. The Add-Fare funds are consumed at the appropriate tap-out but no other bits have been seen to change to track a valid-zone state. So how does the ticket record which zones an upgraded ticket is valid for?
Careful readers will have noted that the above test created two Add-Fare transactions. This proves that there are tickets which have no tap-in or tap-out transactions, so one cannot assume that they always do.
Here are two records for a sequence of Add-Fare transactions: A two-zone upgrade of $1.25 followed by an upgrade of $1.50 to three-zones. Note the different transaction types (which is disconcerting). Also, the Stored Value is cumulative: The second transaction does not say "add $1.50" it says "the stored value is now $2.75"; thus, the transaction records are not context-free. This system was either not designed by people with accounting training, or there weren't enough bits for proper accounting practices.
040000FF823E0400FF000000490221F8 ← Transaction Record, add $1.25
0400 ← Time (0x0000: 11 bits 0x000 → 00:00; 5 bits 0x04 → Add-Fare(?)
00 ← Date Offset (0x00 → what?)
FF ← Unknown; Zones not determined?
823E04 ← (0x043e82: 6 bits unknown; 11 bits stored value; 7 bits Sequence Number)
Unknown = 0x01, Stored Value = $1.25, Sequence Number = 2
00 ← Minutes since first tap-in; always 0x00 for Add-Fare?
FF ← Expires Offset (this was a bulk ticket upgrade)
0000 ← Location Code; always 0x0000 for Add-Fare?
00 ← Line Code; always 0x00 for Add-Fare?
4902 ← Machine Code (0x0249 == 01111 → Waterfront Stn TVM)
21F8 ← MAC
000000FF83890400FF00000049023AD6 ← Transaction Record, add $1.50
0000 ← Time (0x0000: 11 bits 0x000 → 00:00; 5 bits 0x00 → Purchase(?)
00 ← Date Offset (0x00 → what?)
FF ← Unknown; Zones not determined?
838904 ← (0x048983: 6 bits unknown; 11 bits stored value; 7 bits Sequence Number)
Unknown = 0x01, Stored Value = $2.75, Sequence Number = 3
00 ← Minutes since first tap-in; always 0x00 for Add-Fare?
FF ← Expires Offset (this was a bulk ticket upgrade)
0000 ← Location Code; always 0x0000 for Add-Fare?
00 ← Line Code; always 0x00 for Add-Fare?
4902 ← Machine Code (0x0249 == 01111 → Waterfront Stn TVM)
3AD6 ← MAC
Note that Add-Fare transactions for a ticket upgrade contain neither a timestamp nor a Date Offset (all zeroes). It's unknown why this is. An implication is that one can't time-sort all transactions, which isn't actually important unless there are enough transactions to wrap the sequence number and one is maintaining a complete history of transactions external to the ticket.
It seems that tickets cannot be upgraded after the first tap-in. This suggests a potential problem if one makes the obvious (and common?) mistake of travelling into another zone with a one-zone ticket. What happens in practice is that a fare gate will reject a tap-out attempt, as expected, and indicate that money is owning. The Exit TVM can then be used to load the required Add-Fare, which the fare gate will then deduct on tap-out.
However, the record for this Add-Fare transaction is different than for the Add-Fare transaction of a ticket upgrade. Specifically, it has a non-zero timestamp and Date Offset but the timestamp and Location Code are the same as for the previous transaction entry, not for when and where the transaction actually happened. What is correct is the Machine Code, so this situation is not undetectable.
Here's a record of one of these transaction records.
F46402FF833E0000013900092B163BAE ← Transaction Record, add $1.25
F464 ← Time (0x64f4: 11 bits 0x327 → 13:27; 5 bits 0x14 → AddFare (at exit TVM?)
02 ← Date Offset (TVM purchased, not a bulk ticket)
FF ← Unknown; Zones not determined?
833E00 ← (0x003e83: 6 bits unknown; 11 bits stored value; 7 bits Sequence Number)
Unknown = 0x00, Stored Value = $1.25, Sequence Number = 3
00 ← Minutes since first tap-in; always 0x00 for Add-Fare?
01 ← Expires Offset
3900 ← Location Code (0x0039 == Oakridge)
09 ← Line Code (0x09 == Canada Line)
2B16 ← Machine Code (0x1162b == 71211 → Sea Island Exit TVM)
3BAE ← MAC
Here's an enumeration of the "errors" in this transaction.
A ticket has a limited number of transaction records (two!) so there needs to be a way to carry forward the amount of time that the ticket has been active so that the remaining transfer time can be determined. This appears to have been done by recording the number of minutes (since the ticket was first tapped) in each transaction record as it is created.
This is good in theory but the implementation has some quirks. Observation suggests that this counter normally counts up to 90 minutes but will record a value in excess of that for one transaction, presumably to preserve accuracy when the next tap-out occurs after that limit. However, the next tap-in transaction (such as is possible after 90 minutes with a DayPass) will record a zero, implying a loss of accuracy; however, it may be possible to look at the timestamp of the previous tap-out transaction to obtain the time delta between the transactions.
The largest value seen to date is 204 minutes (proving that this field is at least eight bits long) in a One Zone ticket, recorded during a normal weekday afternoon. How the ticket was able to tap out after three hours in-system is a mystery! Should this not have been an Exit Ticket situation? There are no Add-Fare transactions in the ticket.
A discarded Two Zone ticket was found with an extremely long time of 353 minutes between tap-in and tap-out. Should this not also be an Exit Ticket situation? The value for the minute counter is 0x61; together with the above ticket data, this proves that the counter is eight bits long and wraps after 0xff, as 353 - 0x100 = 0x61.
There appears to be no method to detect that this wrap has happened, so this counter is not reliable over the life of a ticket. In practice, it should be all right for a short-term ticket (which shouldn't last for anywhere close to 255 minutes) but it's unusable for DayPasses that use much travel time. Perhaps TransLink (or Cubic?) tracks each tap in the back-end servers to determine total in-system time in the case of a long-lived ticket.
Warning: This is pure speculation but agrees with all observed tickets and makes sense; Occam's razor!
The Transit Tariff lists a product called a MultiPass. The description is lacking precise details, so the author can only assume that it is a day pass that is usable for a certain number of contiguous service days once activated by a tap-in; for example, a ticket that is good for a week of service. The obvious questions are: 1) How does a MultiPass ticket store the initial number of service days, and 2) how are they tracked as they count down? The answers are not known to the author but intelligent guesses can be made from observations of data from non-MultiPass tickets.
Comparing used bulk tickets to TVM tickets proves that there is a relationship between offsets 0x02 and 0x08 of the Transaction Record, as the difference is always 1 after a bulk ticket has been activated by its first tap-in. This same difference was seen with TVM tickets but no intelliegence could be gleaned from that difference by itself.
As no MultiPass ticket has yet been examined by the author, it really would be pure speculation to ponder how the number of service days might be stored in an unactivated ticket. But that's no impediment! Some guesses are:
An untapped MultiPass needs to be seen before an answer to this question is possible.
The second guess, above, is the simplest theory that accounts for the observed relationship between offsets 0x02 and 0x08, and generalizes support for multiple contiguous service days. This assumes that the value at offset 0x08 for an activated ticket simply stores the expiry date, as referenced to the Date Code like the Date Offset is. Thus, we can tentatively define offset 0x08 as the Expires Offset because it likely indicates the expiry date of the ticket.
This theory proposes that an Expires Offset value of 0xff indicates that a ticket has not been activated, and that any other value indicates the day on which the ticket is expired. From this, the rules to determine the expiry date of a ticket are very simple.
A fare product is subject to other expiries such as transfer time and in-service time but that is different from the "is this ticket valid for use today" expiry indicated by the Expires Offset.
Careful readers will note three things.
Of course, the only way to prove any of this is to find a few used MultiPass tickets or obtain an unused one and use it. The tariff's description of who is eligible for a MultiPass suggests that there will never be large numbers of such tickets created, much less able to be found as transit litter! If anyone has one and could scan it via NFC, then please send the data along as you use the ticket!
There are two ticket types known to be defined. The storage is a byte so it's either eight bits or a number; either way, it seems that many more types could be defined.
After most of the ticket data format seemed to be understood, there still appeared to be no way to differentiate a tap-in from a tap-out. There are examples of tickets with tap-in and tap-out records within the same minute which are almost identical but the differences were in allegedly understood fields unrelated to anything such as tap direction. It was speculated that all transactions with an odd sequence number were tap-out transactions but there are examples of tickets with two bus taps. Understanding improved with further analysis.
Analysis of tap-in and tap-out records suggested that the tap direction was encoded in the 16-bit time: All tap-in transactions examined were found to be recorded with a time at 41 seconds past the minute, and all tap-out transactions were similary recorded at 11 seconds (as were bus taps). All purchase transactions examined were found to be recorded to the minute, at zero seconds.
After determining that what was believed to be a 16-bit time of day was actually an 11-bit time plus a 5-bit field for something else, it became clear that the something else is actually the transaction record type. This implies that it would be very unlikely that transactions would ever be recorded with anything better than one-minute precision.
The types of transaction records seen in tickets are listed here.
The type of product loaded onto a ticket appears to be determined by a single byte. No tickets have yet been discovered that would suggest a multi-byte code.
The observed values and the type of product are:
It's possible that the Product Code is a 7-bit number. There's the precedent of the Sequence Number, and removing the most significant bit from the above codes produces much more logical numbers, starting at 1 instead of 0x81. If this is correct then the most significant bit is another unknown.
Offset 0x0b of the Product Record is an irritating mystery because it stands out in a sea of zeroes but has no obvious purpose. It increments by six for each ticket vended, wraps after 0xff, and the value is an even number for every ticket found. The author speculates that the purpose of this byte may be to make attacks on the MAC more difficult.
This appears to encode the type of line or vehicle where a tap took place. The values observed and apparent meanings are:
An examination of the Transit Tariff suggests that there may be another Line Code for Bowen Island. The author speculates that West Vancouver Blue Bus may also have its own Line Code as it's operated under contract to TransLink. Can anyone from those areas supply NFC-scanned data from used tickets?
This number appears to encode the location where the ticket was tapped. SkyTrain stations appear to have values less than 128, whereas buses all appear to be greater. The SkyTrain station codes are well-known but the bus codes appear not to be directly convertable to a stop number without further internal data from TransLink or Cubic.
Analysis of the code numbers from bus taps shows sections of one-to-one correspondence between the codes and the stop number with both incrementing in step but there are discontinuities which appear to make it impossible to convert a code to a stop number mathematically. In other words, a lookup table is needed that is not known to be publicly available.
Here is a CSV file of the known SkyTrain and SeaBus stations.
Warning: This is mostly speculation!
This byte appears to have a relation to the zone system, thus the tentative Zone Code name; however, when outbound Canada Line trains enter Zone 2 at Bridgeport Station this byte does not change. This prevents the number from being easily mapped to Zone 1, 2, or 3. Is this a configuration error with Canada Line stations? That would seem to be very unlikely as it should have been detected already, if so.
Observed values:The value 0x06 has only been seen in Transaction Records on weekends or after 18:30; strongly suggestive of a relation to zones!
This byte requires much more study.
Many (all?) Compass devices have a five-digit ID number. These numbers have a structure: The thousands digits correspond to the SkyTrain station number, the hundreds digit is used to classify the equipment's function, the tens digit may be a floor, area, or group number, and the ones digit is a unit number.
This ID number is stored in the ticket's Transaction Records when they are created. The values observed for the hundreds digit in tickets or on actual device labels are:
The gate number can be difficult to spot because it is printed on a metal-coloured label that's blocked by a gate paddle at most viewing angles. Look at the vertical member on the left side behind a closed gate, halfway up from the floor.
It has been observed that the Machine Code word from a ticket does not always match the number on the gate's label. Sometimes the number appears to have no relation at all; are some gates misconfigured, or is such a number the key for a database lookup?
The ID of devices at stations numbered 65 or less can be stored exactly in a 16-bit word. For other stations the device number is truncated to 16-bits to fit in the Machine Code word.
For tap-in or tap-out Transaction Records this isn't a problem because the Location Code is part of those records, allowing for one to test if a reversal of the truncation (by adding 0x10000) is correct by checking for a matching Location Code. It is a problem for purchase Transaction Records because the Location Code is not written to those records. The best effort in that case is to check if the hundreds digit of the Machine Code is a "0", "1", or "2" after adding 0x10000; however, this is not infallible, as there are a few extant tickets which may have been created by a special device as they do not have a normal TVM number but one that appears to be a database key.
The author calls this a MAC but it's not known if it's a cryptographically secure MAC, a CRC, or a checksum, as the algorithm is not publicly known. What is known is that a TVM will reject a ticket that has had as little as one bit of Compass Data changed, or if complete but unmodified Compass Data is copied from one ticket to another.
After the time representation was believed to be understood but before it was realized that the five least significant bits were not used to store seconds, it was found that some tickets had transaction records that appeared to travel 30 seconds backwards in time. Huh? Well, each transaction record has a sequence number that monotonically increments for each new transaction (but presumably wraps after 0xff) and, logically, one would expect that the time would also increase monotonically (but wrap at 0xb400, the encoding of 24:00:00) but that is not always true, as testing to cause the Time Travel effect demonstrated.
When this effect was discovered it was also noticed that the times of both transaction records were always in the same minute. On the theory that Compass was creating two transactions at once to correct some error situation, attempts were made to force Time Travel to occur by deliberately not tapping in or not tapping out (via use of an open gate) to test what Compass would do at the next tapping event. These tests were successful and revealed that there was also a forward Time Travel of 30 seconds which was not initially noticed because both the sequence number and time for that move in the expected direction.
Here's a real ticket that shows 30 seconds of backwards time travel. The two transaction records have sequence numbers 0x04 and 0x05 yet their times are 0x0cb6 and 0x0ca6, respectively: The sequence number increments but the time decrements.
0481707D8A7743803E48000000000000 0A04004F200182000000009A0000F8F1 A60C020605000032014100093C822423 B60C020604000032014100093C823EC9
So what's the explanation? Well, anyone who remembers reading about Tap Direction above might have an insight into what's going on. If one has skipped a tap-in and Compass notices an insane state on the next tap-out then it will create two transaction records but the order of creation results in the apparent time travel: A fake tap-in transaction will be created first (next expected sequence number and recorded as 41 seconds into the minute), making up for the missed tap-in. Then a tap-out transaction will be generated (next expected sequence number and recorded as 11 seconds into the same minute): Instant 30 seconds of backwards time travel.
The falacy, of course, is that those bits don't represent seconds, thus there is no time travel effect as Compass Time is only to-the-minute. Still, it was an amusing exercise to explore this effect; it's things like that which make research fun!
As noted above, the author would love to have a smartphone app that addresses the information shortcomings inherent to an NFC card that can't have ephemeral or changing items like variable expiry times printed on it. The author's opinion is that it's de rigueur for every transit system using NFC in these modern times to have such an app.
I imagine that making Compass friendly to third-party development could be both desirable and undesirable for TransLink or Cubic. It's desirable because TransLink avoids the work of writing, maintaining, and providing end-user support for an app; only data needs to be released: Some documentation, some lookup tables, and a public read-only key for the Compass Card. It's undesirable if neither TransLink nor Cubic want to release that info, although there should be no security holes opened by being able to read a Compass Card or Ticket – being able to write is what could cause trouble.
There is precedent for third-party apps: TransLink has downloadable data available for third party smartphone apps to display routes, bus stops, scheduling, etc. TransLink has not released its own smartphone app for those tasks but has left the market to third party developers, which have produced more than a few apps.
In the interests of inspiring a good app that would serve the general public well and satisfy this technical user, an app should have this minimum set of functions for Compass Cards and Compass Tickets.
A warning would be appropriate to disclaim the possibility of online updates that have not made it to the card yet.
A number of proofs-of-concept app releases were made beginning in April
2016 based on the above analysis of tickets. The current release is
now better than proof-of-concept quality and is suitable for both
regular and technical users. It is available
here
as an Android apk file or on the
Play
Store as well.
Copyright © 2016 Toomas Losin. All rights reserved. This document is released under the Creative Commons Attribution (CC BY) license.
| Copyright © 1996-2017 Toomas Losin | Contact | Disclaimer | ||