Handling 29-Bit (Extended) Format

[mike_b]

Based on some research online, I suspect my car is using 29-bit IDs (as opposed to 11-bit). A few questions:

1. Is there any easy way to confirm this?

2. How do I go about handling message requests and responses differently for 29-bit? I know CANMessage has an "extended" property, i.e.

message.extended = true;

...but what else needs to be handled differently when constructing the message and parsing the response? For example, would the Message ID for retrieving PIDs be different from the usual 0x7DF?

Thanks,
Mike

[jvanier]

Both 11-bit and 29-bit messages can coexist on the same bus.

Since the PID messages are legislated the 0x7DF messages should be the same.

message.id is a 32-bit value. When receiving a message with an 11-bit ID, only the lower 11 bits will be filled in and message.extended will be false. When receiving a message with a 29-bit ID, the lower 29 bits will be filled in and message.extended will be true.

While you could have messages with the same 11-bit and 29-bit ID on the same bus (0x110 and 0x‭00000110)‬ or 29-bit IDs smaller than 0x7FF, that would be super confusing and I hope no car manufacturer does this!

So if you just print the IDs you should be able to see if you have 29-bit IDs in there. You can also print message.extended to be sure.

[mike_b]

Julien,

Thanks! Can you please clarify on this:

Since the PID messages are legislated the 0x7DF messages should be the same.

Does this mean that even if the vehicle uses 29-bit messages, that I should still be able to send the usual diagnostic requests to 0x7DF and expect to get a reply on 0x7E8? This is part of my struggle where I'm trying to figure out why I'm not getting anything back on 0x7E8 (or one of the neighboring addresses).

Thanks,
Mike

[jvanier]

Yes. You should still be able to to send the usual diagnostic requests to 0x7DF.

Try different CAN speeds. Maybe your car doesn't use the usual 500 kbit/s. I'd try 250 kbit/s and 1,000 kbit/s.

Change it in setup() with carloop.setCANSpeed(250000); before carloop.begin();

[SinusoidDelta]

Sorry if I'm a bit off topic. Is there some way to detect the baud rate programmatically? Also can you access both low speed and high speed layers simultaneously?

[cyclin_al1]

Programmatically, you would probably have to cycle through different baud rates until you get a valid response. Just be sure to wait long enough for a timeout between trying different speeds, or you might miss the response.

As for accessing both low speed and high speed layers simultaneously, are you referring to pins 6 & 14 on the connector (see link)?
https://en.wikipedia.org/wiki/On-board_diagnostics#OBD-II_diagnostic_connector.

Yes, carloop is connected to both these pins. As for what the existing firmware will do, or what could be possible if firmware was written would be better answered by @jvanier.

If you are referring to primary and secondary CANbuses, then you would need to develop another version of carloop hardware (call it carloop-multi?) that uses the electron (since it has a second CAN channel) with a second CAN transceiver chip in the hardware.

I am working on something similar (calling it carloopRetro) that has both a CAN transceiver and a K-Line transceiver. However, I doubt that CAN and K-Line would ever be used at the same time.

[mike_b]

@jvanier,

I know for sure the car is on 500 kbits because it says so in the manual (and the other bitrates don't return any messages).

Using an OBD scan tool I found out today that the ECM on the vehicle actually has an ID of 0x18DAF111, which would indicate that I should expect replies using this 29-bit message ID. I googled this ID and found some other hits where this ID was used as the reply ID for Honda vehicles.

So with that in mind, how would I figure out the broadcast ID where I need to send the request? Am I still supposed to send the request to 0x7DF and then expect a reply from 0x18DAF111? Or do I send the request to 0x18DAF111?

Thanks,
MIke

[alanm]

n mind, how would I figure out the broadcast ID where I need to send the request? Am I still supposed to send the request to 0x7DF and then expect a reply from 0x18DAF111? Or do I send the request to 0x18DAF111?

Hi @mike_b, I want to clarify something regarding CAN Message IDs. When sending CAN messages on the bus, all modules that are connected to that bus will be able to receive the CAN message you are sending. Similarly, when you connect Carloop to the OBD-II port you can see CAN messages being broadcasted by other modules on the bus.

People sometimes confuse the Arbitration ID (0x7DF in this case) being tied to module or node. That is simply not the case, since many modules can send and receive CAN messages with same IDs. The confusion stems from the practice in the 90s to assign CAN IDs to the modules but as the CAN bus wikipedia article explains that practice is no longer valid:

In the early 1990s, the choice of IDs for messages was done simply on the basis of identifying the type of data and the sending node; however, as the ID is also used as the message priority, this led to poor real-time performance. In those scenarios, a low CAN bus utilization of circa 30% was commonly required to ensure that all messages would meet their deadlines. However, if IDs are instead determined based on the deadline of the message, the lower the numerical ID and hence the higher the message priority, then bus utilizations of 70 to 80% can typically be achieved before any message deadlines are missed.

In other words, you should send request 0x7DF (with the appropriate data e.g. 01100011) and see a response from the ECM as a new message with Arbitration ID 0x18DAF111 and its corresponding data. It has nothing to do with the sending / receiving node. Hope this clarifies some of the confusion surrounding CAN.

[mike_b]

@alanm,

Thanks for the clarification! Let me paste a bit of code in here and I would like you to let me know if this code should work, given the discussion above.

Filtering (for now, just include anything with an extended message format); this runs before calling canDriver.begin()

canDriver.addFilter(0, 0, CAN_FILTER_EXTENDED);

Request message construction:

CANMessage message;
message.id = 0x7DF;
message.len = 8; // message will always be 8 bytes
message.data[0] = 0x02; // 2 bytes of data in message
message.data[1] = 0x01; // Get current data
message.data[2] = 0x0C; // The PID to retrieve (Engine RPM)
message.data[3] = 0x00;
message.data[4] = 0x00;
message.data[5] = 0x00;
message.data[6] = 0x00;
message.data[7] = 0x00;

if (!canDriver.transmit(message))
{
  Serial.println("Unable to transmit message!");
}

The below code runs in a loop that sleeps for just 10ms between loops:

CANMessage replyMessage;
int messageCount = canDriver.available();

if (messageCount > 0)
{
  while (canDriver.receive(replyMessage))
  {
    // Got an extended message; light up the LED
    digitalWrite(ledPin, HIGH);
  }
}

So you can see that the above code is pretty forgiving and doesn't even look for a reply specifically with the ID of 0x18DAF111 (just looks for any extended message). I still can't get a reply though (i.e. blue LED doesn't light up). What am I not doing correctly?

Thanks,
Mike

[cyclin_al1]

@mike_b, I wonder if you need a strategic check for start of receiving in your code? It is difficult to tell from posting just a snippet, but there is one thing I wonder about.

while (canDriver.receive(replyMessage))

I wonder if your code checks to see if it receiving before the receiving actually starts. If that is the case, your code will exit the loop before receiving (which is unintended), instead of exiting the loop after receiving is complete (which is what I think you want).
What I would suggest is putting in a delay or some sort of if/while to see that receiving actually gets started. Once you know it is started, then you can check to see when it completes receiving.

One other thing to note, which is a Particle thing, not a carloop thing.
If for some reason, the receiving is not completed within a reasonable time, you will be stuck in that loop for too long. This is called blocking code. Particle devices need time once in a while to keep the connection alive (WiFi, Cellular or BLE) and to do some general housekeeping. If you block for too long, you will lose your connection and then your code will behave strangely. I recommend using a timeout to exit from loops, and then you can handle the timeout error OR you can use a SYSTEM_MODE (if I remember correctly) that puts your code and the Particle code in separate threads.

[mike_b]

@cyclin_al1,

Thanks for the suggestions-- regarding the Particle system mode stuff, I already have that covered (system mode MANUAL with system thread enabled).

This comment is very interesting though:

I wonder if your code checks to see if it receiving before the receiving actually starts. If that is the case, your code will exit the loop before receiving (which is unintended), instead of exiting the loop after receiving is complete (which is what I think you want).

I'm not totally sure what you mean here and I don't recall seeing any sample code anywhere that does anything special with the receive() call beyond what I'm already doing. Can you please provide a simple example with code/pseudocode to clarify this point? You mentioned to put something like an if/while "to see that receiving actually gets started" but I don't understand how that translates to code.

Thanks,
Mike

[cyclin_al1]

@mike_b, sorry for the delay. I can see the forums but not post when I am at the office.

Pseudo-code for what I am thinking is this:

serial.print "waiting for receive to start" // debug messages to USB serial
 while ( NOT canDriver.receive(replyMessage) OR timeout < 5 sec)
{ } // wait for receive to start
if (timeout reached 5 sec)
{
serial.print ("receive did not start within 5 sec")
} else {
serial.print ("receive started")
 while ( canDriver.receive(replyMessage) OR timeout < 5 sec)
{ } // receiving
if (timeout reached 5 sec)
{
serial.print ("receive did not complete within 5 sec")
} else {
serial.print ("receive successful")
}

You may not need this in the final code, but this could help out with experimenting and debugging. If you monitor serial over USB, you will be able to see how far the receiving gets before you run into problems. You can also take this pseudo-code and add in some additional functionality to do re-tries.

Also, do you have an oscilloscope or logic analyzer to monitor the message receiving? You could probe D2 (CAN_RX) on the Particle device (not sure if it is Photon or Electron or a compound) to see if there is signal activity on receiving. Also, you could probe pin 6 of the OBDII connector, but you would need a logic analyzer to tell the difference to sending out the request versus receiving a response.

[mike_b]

@cyclin_al1,

Thank you for the clarification and additional ideas! I figured out the problem and I have a very interesting explanation.

I have one of those generic scan tools that you can plug into the OBD port and check/reset trouble codes, see live data, etc. The scan tool works perfectly fine when I plug it in to my car. It reads live data like Vehicle Speed which the Carloop was unable to do. So I wanted to figure out why the scan tool worked and the Carloop didn't.

I bought a simple OBD Y-splitter and connected the scan tool to one end and the Carloop to the other, so I could use the Carloop to monitor messages being sent to and from the scan tool. I discovered a few interesting things from this:

• I need to use message ID 18db33f1 to send the request; 7DF does not work
• The response has a message ID of 18daf111 or 18daf11d (probably depends on the responding ECU)
• The last 5 bytes in the message data must be populated with a specific set of values; if these are set to 00 or 55, you will not get a reply; I won't paste the values in here in case this is some kind of proprietary key not meant to be released to the public

After I made those adjustments, the Carloop worked perfectly to retrieve Vehicle Speed and RPM (didn't try anything else). Hopefully this post has a few tips that help others. Thanks to everyone for your help!

Mike

[cyclin_al1]

@mike_b, Great sleuthing to figure out what is going on in your car!

@alanm, Any ideas what is going on with this solution?

It appears to violate the legislated OBDII requirements (at least in North America). On the other hand, could you be plugged into a CANbus other than the one used for OBDII. My only wild guess is that some module is sending on a secondary bus to another module, and then that second module forwards to the OBDII CANbus.
Then again, I am focusing on the older K-Line, so I could have missed some new development...?

[alanm]

I won't paste the values in here in case this is some kind of proprietary key not meant to be released to the public

no need to worry, @mike_b! Luckily we are allowed to peek under the CAN bus hood all we want https://www.wired.com/2016/10/hacking-car-pacemaker-toaster-just-became-legal/

[alanm]

I need to use message ID 18db33f1 to send the request; 7DF does not work

Yes, that is correct. @cyclin_al1, in response to your question, based on the ISO 15765-4 Standard for CAN messages, that request ID is required for CAN buses with extended (29-bit) format. See table from ISO standard:

For 11 bit identifiers, the correct CAN IDs are:

For those of you trying to learn more about Diagnostics on
Controller Area Networks (CAN), this standard goes into quite a bit of detail: https://sourceforge.net/p/rusefi/tickets/_discuss/thread/65c5390c/5610/attachment/iso_15765-4.pdf

[mike_b]

Thanks @alanm! Very helpful document.

[cyclin_al1]

Thanks @alanm, lots of great detail!

[egarl004]

Running into similar issues here with a Honda Pilot. Anyone have some example code? I've changed my message ID but could use some pointers with the rest.

[cyclin_al1]

@mike_b, did you get your code to a point where it could be used as a complete example, or have some good pointers for @egarl004 ?