I2C R/W Test. Debug code posted here.

This forum is dedicated to software development related to MultiWii.
It is not the right place to submit a setup problem.

I2C R/W Test. Debug code posted here.

Postby mr.rc-cam » Thu Nov 10, 2011 9:42 pm

Nov-12-2011 UPDATE: Alex's V1.9 released on Nov-11-2011 now has an I2C error counter in the GUI's debug2 window. Please use it to determine if your MultiWii has I2C performance problems. Users with earlier versions (V1.8 and higher) can continue to use the test code discussed here to troubleshoot I2C problems.

The initial release for V1.9 had an I2C bug that caused board lockups on some installations. There were several reports of "dead" boards and I was affected too. Although the latest V1.9 releases have fixed the lockup issue, it turns out that the bug helped identify an I2C performance problem on my model that I did not know I had. Furthermore, it is very likely that any board that experienced the lockup issue on the early V1.9 code has an I2C problem. In other words, the lockup would never have occurred if the I2C transmissions were working perfectly.

While debugging the lockup problem I wrote some test code to see what was going on. The test code chirps the buzzer whenever a basic I2C transmission is not successful. I've since added an error counter to it so that the GUI's debug window can show the number of bad I2C hits.

To demonstrate how the test code helped me: After applying the latest V1.9 code fix my model was working fine (and no hints of any I2C issues!). But after applying the test code the buzzer was very noisy (a lot of beeping) which showed there were I2C problems. Temporarily reducing the I2C clock to 100KHz did not help at this point. My o-scope found an inadequate I2C clock signal (SCL); It had poor rise-time and low amplitude. I reduced the SCL pullup resistor value and then ran the test again. The buzzer chirp test was now very quiet, so I thought all the problems were solved. But I allowed the test to run for more than an hour. Every few minutes I would hear a single short chirp. At the end of the hour long test the GUI debug window showed a half-dozen error hits. Reducing the I2C speed to 100KHz eliminated the final remaining sporadic errors and all I2C issues were solved.

To use the debug code all you have to do is edit the sensors.pde file. This code should work in version 1.8 and higher. So open sensors.pde and completely comment out the waitTransmissionI2C() function. Replace it with this:
Code: Select all
// START OF RC-CAM DEBUG CODE.
// I2C Test Code: Used to help identify I2C xfr problems.
// If TWINT flag is not detected the buzzer will chirp and the
// global error counter will increment (max 1000 hits).
// Hint: The GUI's debug window can be used to show the error count.
#define I2C_WAIT_TIME 2000            // Timeout count is long enough to hear the buzzer chirp.
static int16_t i2c_wait_error = -1;   // I2C error counter. Init to -1 for GUI confirmation.
void waitTransmissionI2C() {
  uint16_t count = I2C_WAIT_TIME;
  while (!(TWCR & (1<<TWINT))) {
    count--;
    if (count==(I2C_WAIT_TIME-200)){ // TWINT flag not detected, I2C problem!
        BUZZERPIN_ON;                // Chirp the user.
        if(i2c_wait_error == -1) {
            i2c_wait_error = 1;
        }
        else {
            if(i2c_wait_error<1000) i2c_wait_error++;  // Limit error count to 1000 max.
        }
    }
    else if (count==0) {            // We are in a blocking state => we don't insist
      TWCR = 0;                     // Force a reset on TWINT register
      neutralizeTime = micros();    // We take a timestamp here to neutralize the value during a short delay after the hard reset
      BUZZERPIN_OFF;                // Chirp is done.
      break;                        // Abort, this xfr is bad and needs to be killed.
    }
  }
}
// END OF DEBUG CODE


Then open the serial.pde and find this code sequence (may be different if you have edited this file before):
Code: Select all
      serialize16(BaroAlt/10); // 4 variables are here for general monitoring purpose
      serialize16(0);  // debug2


Replace it with:
Code: Select all
      serialize16(BaroAlt/10); // 4 variables are here for general monitoring purpose
      serialize16(i2c_wait_error);  // debug2


Now upload the edits to your wiiCopter and verify that the GUI's debug2 window shows -1. If it shows 0 then your serial.pde edits are wrong. If greater than 0 then you have I2C problems. Pressing the Arduino reset button will clear the count back to -1 if you want to start the test again.

Allow the model to run for an hour. Motors off is best, so you can hear the buzzer (but try it with motors on too to test for intermittent solder connection problems). Basically, if you hear any chirps or beeps then you've got problems. As an alternative, the GUI's debug2 window will show the running error count (max 1000 errors will be counted).

- Thomas
Last edited by mr.rc-cam on Sat Nov 12, 2011 7:36 pm, edited 2 times in total.
mr.rc-cam
 
Posts: 455
Joined: Wed Jul 27, 2011 11:36 pm

Re: I2C R/W Test. Debug code posted here.

Postby copterrichie » Fri Nov 11, 2011 12:43 am

question: was the internal Pull-up Resistors enabled?
copterrichie
 
Posts: 2108
Joined: Sat Feb 19, 2011 8:30 pm

Re: I2C R/W Test. Debug code posted here.

Postby mr.rc-cam » Fri Nov 11, 2011 1:32 am

question: was the internal Pull-up Resistors enabled?

Yes and no. I tried both ways and neither changed the problem.

My custom IMU has FET based logic level conversion that includes all the necessary pullups (10K ohms). My scope showed that SDA signal was fine, but the SCL was lazy looking (shark fin rising edge profile, reduced amplitude). To satisfy my curiosity I enable the ATMEGA's internal I2C pullups to see if that would help improve the SCL since it would be similar to adding a resistor in parallel with the external pullup (effectively reducing its ohm value which is something that was needed in my particular installation). The scope showed a minor improvement but I still had some I2C errors.

So I reduced the external SCL pullup resistor's value on the CPU/5V logic side and that cleaned up the signal to perfection. Now the SCL has very square edges and robust logic levels.

And it rewarded me by fixing an annoyance that haunted my Quad; I used to get a gyro data glitch about 3-5 times a minute with V1.8 and V1.8p2. After fixing the SCL signal I reloaded V1.8p2 and the glitch was gone. So a 2 cent resistor was all that I needed to eliminate an old mystery.

Long story short, I don't think I would have been able to find the problem and confirm the hardware fix without the test code. So I figured someone else may find a good use for it. ;)
mr.rc-cam
 
Posts: 455
Joined: Wed Jul 27, 2011 11:36 pm

Re: I2C R/W Test. Debug code posted here.

Postby mickt » Fri Nov 11, 2011 12:52 pm

mickt
 
Posts: 3
Joined: Fri Nov 11, 2011 12:48 pm

Re: I2C R/W Test. Debug code posted here.

Postby mr.rc-cam » Fri Nov 11, 2011 5:34 pm

mickt wrote:I found this may be of interest http://dsscircuits.com/articles/effects ... stors.html

That article nicely demonstrates the issue that affected my installation, which has six I2C slaves devices. With the 400KHz I2C clock my waveform looked like the one shown with 10K pullup/400KHz. After I reduced the pullup to 2.2K and changed the clock to 100KHz it looked like the 2.2K/100KHz waveform posted in the article.

Keep in mind that the debug code is a general purpose I2C test. If your model fails it then that does not mean you have the wrong pullup resistors. Finding the cause of the failures is up to your troubleshooting skills, but knowing you have I2C problems is half the battle.
mr.rc-cam
 
Posts: 455
Joined: Wed Jul 27, 2011 11:36 pm

Re: I2C R/W Test. Debug code posted here.

Postby Noctaro » Fri Nov 11, 2011 7:01 pm

Ok,
it looks like i found my problem too!
As far as i know the Paris V3 board got 2,2k resistors integrated (activated on PARIS by closing the solder pads). I bought the preasembled version with BMA180 + LLC and added HMC5883l + BMP085.

As right now i don´t want to disassemble my copter, give me a hint please.

For this Paris board, the solder bridge should be closed already. Or do i get something wrong?
If the bridge is closed, do i need to activate the internal pullups in the software?

Right now i got them enabled and i think this may cause the glitches. I read anywhere that internal and external PullUps, if both are activated will create bad data.
I did not dare to deactivate the internal pullups right now, as i am afraid of destroying my sensors.

thx & greetz
Noc
Noctaro
 
Posts: 251
Joined: Thu Sep 08, 2011 11:15 am

Re: I2C R/W Test. Debug code posted here.

Postby mr.rc-cam » Fri Nov 11, 2011 7:16 pm

As far as i know the Paris V3 board got 2,2k resistors integrated (activated on PARIS by closing the solder pads).

If it has the Sparkfun style LLC then the default pullups on the LLC BOB are 10K ohms. You will have to visually inspect your Paris board to determine its pullup values and to see if they are enabled. But unless you know that they are sourced from 5V and are on the "high voltage" CPU side, then I think it is best to disable them. If you need more help with this then post a detailed schematic to the V3 board.

I bought the preasembled version with BMA180 + LLC and added HMC5883l + BMP085.

I suggest you remove the two devices you added (unsolder their SCL and SDA wires), disable them in the sketch, and try the test again. And reduce the I2C speed to 100KHz. If you have an o-scope then use it to see the quality of your I2C signals.

If the bridge is closed, do i need to activate the internal pullups in the software?

With the typical LLC (like Sparkfun's) there will be no harm if the CPU's internal pullups are enabled. But normally they would be disabled when external pullups are used.

Note: There are many reasons for the I2C test to fail. So keep in mind that your chosen pullup values may be fine and the problem might be elsewhere.
mr.rc-cam
 
Posts: 455
Joined: Wed Jul 27, 2011 11:36 pm

Re: I2C R/W Test. Debug code posted here.

Postby KeesvR » Fri Nov 11, 2011 8:08 pm

mr.rc-cam wrote:
To use the debug code all you have to do is edit the sensors.pde file. This code should work in version 1.8 and higher. So open sensors.pde and completely comment out the waitTransmissionI2C() function.


- Thomas

Can you exactly describe what i need to comment out, I looked in the sensor.pde and I'm not sure what i need to do.
I'm a noob at coding. :?
KeesvR
 
Posts: 194
Joined: Fri May 27, 2011 6:51 pm
Location: The Netherlands

Re: I2C R/W Test. Debug code posted here.

Postby mr.rc-cam » Fri Nov 11, 2011 9:03 pm

KeesvR wrote:Can you exactly describe what i need to comment out, I looked in the sensor.pde and I'm not sure what i need to do.
I'm a noob at coding. :?


(1) Make a backup of sensors.pde and serial.pde to another directory (copy them to the desktop). The backups will allow you to revert back to the stock code after you've performed the test.

(2) Open sensors.pde and find the waitTransmissionI2C() function. It will look something like this (may vary a tiny bit depending on the sketch version):
Code: Select all
void waitTransmissionI2C() {
  uint16_t count = 255;
  while (!(TWCR & (1<<TWINT))) {
    count--;
    if (count==0) { //we are in a blocking state => we don't insist
      TWCR = 0;  //and we force a reset on TWINT register
      neutralizeTime = micros(); //we take a timestamp here to neutralize the value during a short delay after the hard reset
      break;    // Abort.
    }
  }
}


(3) Delete that code and replace all of it with the revised waitTransmissionI2C() function code. That is to say, cut and past the entire code segment published in post #1. This code begins with "// START OF RC-CAM DEBUG CODE." and ends with "// END OF DEBUG CODE."

(4) Open serial.pde and perform the small edit there, as explained in post #1.


- Thomas
mr.rc-cam
 
Posts: 455
Joined: Wed Jul 27, 2011 11:36 pm

Re: I2C R/W Test. Debug code posted here.

Postby KeesvR » Fri Nov 11, 2011 9:30 pm

All clear now, the first step i had done already, the second was confusing for me because i found a lot of "waitTransmissionI2C" lines.

Thanks Thomas, I'm gonna try this.
KeesvR
 
Posts: 194
Joined: Fri May 27, 2011 6:51 pm
Location: The Netherlands

Next

Return to Software development

Who is online

Users browsing this forum: Exabot [Bot], Google [Bot], shikra and 5 guests