Inflight data logging HW+SW

User avatar
EOSBandi
Posts: 802
Joined: Sun Jun 19, 2011 11:32 am
Location: Budapest, Hungary
Contact:

Inflight data logging HW+SW

Post by EOSBandi »

Hi Guys,
While working on GPS and mag, I realised that i need some onboard data logging for getting neccessary data to analise and debug control algorithms. Telemetry is not a viable option, because it has limited bandwith. So I started to implement an arducopter style onboard dataflash logging. It needs an SPI port plus two other free pins. (Sorry, no Atmega328). I'm attaching PCB plans for the dataflash module (single sided, smd). Chip is the atmel AT45DB321D which has 8192x512byte capacity (4 megabyte) and two sram buffers to maintain high speed writing.
On the software side I do not want to reinvent the wheel, so planning to use arducopter's dataflash library.

Questions :
Anybody interested in it?
If yes, what data do you want to see in the log?
Is it OK to have a command line interpreter like shell, for manipulating logs? (btw, and other parameters ?)
Dataflash handling is currently implemented as a C++ library, since this function will work on AtMega1280 and 2560 based boards only, is it ok to use this type of library ?
Attachments
DataFlash-blob.zip
Eagle files for the dataflash module
(15 KiB) Downloaded 299 times

noobee
Posts: 66
Joined: Fri Feb 25, 2011 2:57 pm

Re: Inflight data logging HW+SW

Post by noobee »

why not log through the serial port at 115200 to a serial logger? like this one: http://www.sparkfun.com/products/9530

is the bandwidth still insufficient?

User avatar
EOSBandi
Posts: 802
Joined: Sun Jun 19, 2011 11:32 am
Location: Budapest, Hungary
Contact:

Re: Inflight data logging HW+SW

Post by EOSBandi »

Yes.
At 115200 we have about 10 char/ms but that pretty much saturates the serial line, which is bad for inflight logging.
Dataflash has about 50 char/ms for flash writing, but via a sram buffer which filled via a 4Mhz spi bus.

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Inflight data logging HW+SW

Post by Hamburger »

good thing.
Once we have it, the use cases will appear 'like magic'.

Toro480
Posts: 4
Joined: Wed Jul 18, 2012 11:41 am

Re: Inflight data logging HW+SW

Post by Toro480 »

Hi EOSbandi. Is it possible to use your code on the new CRIUS All In One (mega 2560)?
I would like to record data (angle and PID corrections for example) in flash with your method. Did you do some improvement since january?

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

"If yes, what data do you want to see in the log?"

Every single sensor measurement and the resulting state (attitute, altitude) and PID variables MultiWii has derived from them during a flight. It could be at a lower frequency or at full speed ("every single") ;-)

Doing this at 100 Hz over the serial interface at 115200 baud would leave over 100 bytes per log entry ... at 2 bytes per value this seems ok to start. Maybe some kind of MSP command that starts the log process and a #define in config.h to control wether is logs over uart or spi and at full speed or some fixed rate.

Katch
Posts: 280
Joined: Thu Aug 04, 2011 1:44 pm

Re: Inflight data logging HW+SW

Post by Katch »

Flytron does a little data logger - might be worth having a look at his schematic. I'm sure Melih would be willing to help in development.

msev
Posts: 186
Joined: Thu Apr 14, 2011 11:49 am

Re: Inflight data logging HW+SW

Post by msev »

Is logging to SD cards to slow? Could both ways be implemented - the at45 chip + microsd shield ?

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

I have used this OpenLog from Sparkfun in the past, worked great for me and there was very little or no impact on the Cycle time.

http://www.sparkfun.com/products/9530

Sample Text File created with code below.

Code: Select all

2744,1487,1547,1933,1007,1498,1050,1050,1955,1933,2,-8,240,0,0,0,6,-23
2720,1487,1545,1508,1009,1498,1050,1050,1519,1497,2,-8,240,1,0,0,7,-23
2740,1487,1545,1510,1120,1498,1053,1067,1524,1496,2,-8,240,0,0,0,8,-23
2748,1487,1545,1510,1279,1498,1140,1272,1606,1450,3,-11,240,3,1,-2,8,-26
2736,1489,1547,1508,1309,1498,1196,1274,1596,1420,6,-13,234,0,0,0,7,-29
2752,1487,1545,1508,1355,1498,1149,1403,1561,1409,15,-7,229,5,1,1,8,-23
2748,1490,1544,1561,1373,1498,1199,1385,1617,1449,14,-9,231,1,0,0,30,-19


Code segment from version 1.7

Code: Select all

}else if (Datalogging == 1 && DataTimer > 10)
    {
      Serial1.print((millis() - DataStartTime) / 100);Serial1.print(",");Serial1.print(cycleTime); Serial1.print(",");
      for(i=0;i<4;i++){Serial1.print(rcHysteresis[i]); Serial1.print(",");}
      for(i=0;i<2;i++){Serial1.print(motor[i]);Serial1.print(",");}
      for(i=0;i<2;i++){ Serial1.print(servo[i]);Serial1.print(",");}
      for(i=0;i<3;i++){Serial1.print(accSmooth[i]);Serial1.print(",");}
      for(i=0;i<3;i++){Serial1.print(gyroData[i]); Serial1.print(",");}
      Serial1.print(angle[0]); Serial1.print(","); Serial1.println(angle[1]);
     
      DataTimer = 0;
    }
    else DataTimer++;
}

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

I just bought an OpenLog ... logging in ASCII seems to be a waste of bandwidth though, but they claim the don't lose data when sending it bytes at 115200 baudrate ;-)

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

I never had a problem with it nor the speed.

msev
Posts: 186
Joined: Thu Apr 14, 2011 11:49 am

Re: Inflight data logging HW+SW

Post by msev »


copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

msev wrote:Here is a shield for a sd card: http://www.ebay.com/itm/SD-Card-Shield- ... 831wt_1344


You will loose two of the Motor pins using that shield and there has to be software to drive it. The serial version is best for our application in my opinion.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

copterrichie wrote:I never had a problem with it nor the speed.


Because you aren't logging a lot of data (total count of state variables and frequency).

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

Sebbi wrote:
copterrichie wrote:I never had a problem with it nor the speed.


Because you aren't logging a lot of data (total count of state variables and frequency).


LOL, what more data is required to be logged? Good Lord. I suppose if one wanted to, they can write the data in binary but there would have to be an external program to convert it to human readable. :lol:

FYI, I wrote TEXT files that were 500K and larger.

Toro480
Posts: 4
Joined: Wed Jul 18, 2012 11:41 am

Re: Inflight data logging HW+SW

Post by Toro480 »

Thanks for your answers.

@copterrichie : you record datas only every ten time steps. I would like to record datas at each time step.
I have a "logomatic" from sparkfun that I use since many years : http://www.sparkfun.com/products/10216
But limits have been reached for high frequency of the main loop. Limits are also reached for high numbers of datas at each record.
So I'm interested by EOSbandi's method consisting in using flash memory.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

Toro480 wrote:Thanks for your answers.

@copterrichie : you record datas only every ten time steps. I would like to record datas at each time step.
I have a "logomatic" from sparkfun that I use since many years : http://www.sparkfun.com/products/10216
But limits have been reached for high frequency of the main loop. Limits are also reached for high numbers of datas at each record.
So I'm interested by EOSbandi's method consisting in using flash memory.


Interesting, thanks but I don't see the need for that much data logging but I am sure there is a reason.

User avatar
UndCon
Posts: 293
Joined: Mon Feb 21, 2011 2:10 pm

Re: Inflight data logging HW+SW

Post by UndCon »

A small board with SDcard would be cool.

Toro480
Posts: 4
Joined: Wed Jul 18, 2012 11:41 am

Re: Inflight data logging HW+SW

Post by Toro480 »

One possible reason : record sensors values at each time step to replay offline PID control to test another smoothing function or another control method. Compare offline results and the online behavior allow to improve code.
For I term, all datas are needed.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

copterrichie wrote:
Sebbi wrote:
copterrichie wrote:I never had a problem with it nor the speed.


Because you aren't logging a lot of data (total count of state variables and frequency).


LOL, what more data is required to be logged? Good Lord. I suppose if one wanted to, they can write the data in binary but there would have to be an external program to convert it to human readable. :lol:

FYI, I wrote TEXT files that were 500K and larger.


For debugging purposes you'd want to log every sensor value, the derived/smoothed values used in the calculations, etc. ... basically every internal variable there is. And not only every tenth cycle.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

Sebbi wrote:
For debugging purposes you'd want to log every sensor value, the derived/smoothed values used in the calculations, etc. ... basically every internal variable there is. And not only every tenth cycle.


Well, this is your right to believe so but I don't agree. I recorded every tenth value because that is all I needed at the time. I am a very avid student of Statical analysis and what would be better and more accurate is a RANDOM sampling pattern because it would placed the minimum amount of extra load on the processor. Lets assume, every cycle is recorded, there still have to be some filtering and analysis to interpret the data. So recording every value just does not make sense to me but to each their own opinion.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

If you just want to look at log and see what happened that is definetly true. But to use the data to feed improved algorithms (PID) you need every measured sensor value. Since the copter wont "move" any other way in a virtual replay one can argue that this could not be very useful ... but until we have a physically correct copter simulator, it will have to do ... better than crashing a copter while trying it out on the real hardware ;-)

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Inflight data logging HW+SW

Post by Hamburger »

Sebbi wrote:For debugging purposes you'd want to log every sensor value, the derived/smoothed values used in the calculations, etc. ... basically every internal variable there is. And not only every tenth cycle.

I doubt it. Besides you have not clearly stated, what you want to debug.

Looks like you cannot derive too much from that kind of data. The n+1th samples set will be a direct result of implemented algorithm applied to the nth samples set, plus influence from such errors as slowly responding motors. of which you know almost nothing. I am all for a scientific approach, but unless you come up with a working program/simulator which works on the basis of such data _and_ does provide new insight, I will not buy it.

If the intention was to debug the implementation of the algorithm in a kind of replay single step, then moving to another hardware with proper live debugging facilities would do better. Go the timecop route.

Alas, please prove me wrong. Spell the question you want to get an answer for, then follow your approach, present your findings. I love to be convinced by facts.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

Example of some logdata at 10 Hz:

Code: Select all

  0, 0.0, 10
 100, 1.0, 10
 200, 2.0, 10
 300, 3.0, 10
 400, 4.0, 10


The fields are "time since start in ms", "pitch angle in deg", "pitch rotation rate in deg/s". The data is what you expect ... 10 deg/s would cause the pitch to increase 1 degree every 100 ms. But what happens when the sampling frequency of the sensors and the calculations is much higher?

Code: Select all

  0, 0.0, 10
 50, 0.5, 20
 100, 1.5, 10
 150, 2.0, 20
 200, 3.0, 10
 250, 3.5, 20
 300, 4.5, 10
 350, 5.0, 20
 400, 6.0, 10


If you would log this with 10 Hz you would have the following logfile, but clearly it is not what really happened inside the copter during the flight. Can we agree on that? ;-)

Code: Select all

  0, 0.0, 10
 100, 1.5, 10
 200, 3.0, 10
 300, 4.5, 10
 400, 6.0, 10

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

You are aware the I and D of the Process control is TIED to the Cycle time right? Adding LOAD with logging a higher frequency of data will effect the copter's performance thereby misrepresenting what is really happening right? Hmmm. Confused.

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Inflight data logging HW+SW

Post by Hamburger »

We can agree on it being possible to design theoretical constellations.
I asked for real world data and derived new findings.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

copterrichie wrote:You are aware the I and D of the Process control is TIED to the Cycle time right? Adding LOAD with logging a higher frequency of data will effect the copter's performance thereby misrepresenting what is really happening right? Hmmm. Confused.

I am aware of that, that's why I support the 2.2 wish for fixed cycle times so everyones PID values get comparable ;-) But for the calculations going on and the results it doesn't matter how high the cycle time to accurately observe what is happening where ... if it gets too high though, it might get difficult to fly ;-)

Hamburger wrote:We can agree on it being possible to design theoretical constellations.
I asked for real world data and derived new findings.


It is not theoretical. With all the vibrations going on the measurements aren't smooth at all ... one might not miss much when not logging every cycle and in many cases it might be enough to use this data, but I doubt that it will work for PID debugging. Real world data and derived new findings? I didn't have real world data so I simulated it ... I found that the I term in the getEstimatedAltitude() function needs very high I values to have an effect at all and the D term will cause oscillations when too high and finding optimal values for that might need 0.1 changes, but the GUI wont allow that. I want to check that with real world data (OpenLog arrived today *g*) ...

Also, the small angle approximation in the rotateV() function might cause problems. Checking if the deltas are really that small and the rotation has the same results as by using a real rotation matrix would be interesting ... the accelerometer is likely correcting that problem fast enough, but still ... I tried a different way to rotate the gravity vector and noticed that the gyro scale factor is way off too ... these are my theoretical findings so far. Needs accurate logging (or simulation) to confirm ;-)

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

And I state AGAIN, I have had no issues logging data with the OpenLog set to 115200 baud. An ASCII file was good enough for what I needed, if you need more, I am sure you can write binary to a file and decode it as you please. Good Luck.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

copterrichie wrote:And I state AGAIN, I have had no issues logging data with the OpenLog set to 115200 baud. An ASCII file was good enough for what I needed, if you need more, I am sure you can write binary to a file and decode it as you please. Good Luck.


I have to correct myself from earlier ... cycle time will not increase significantly with the current code as serial communication is non-blocking. I have tested the openlog now ... my cycle time is on average around 3000 µs and I can write out 36 bytes every cycle before the serial port (115200 baud) is saturated (it never blocks, but the TX ring buffer will get overridden if more is "sent").

36 bytes every 3 ms mean I can log 18 16-bit variables ... that is not a lot, but it is a start. It could be improved by artificially increasing the cycle time (waiting for the log data to be written/sent instead of fire and forget) or - i guess - with the SPI datalogger EOSBandi proposed when he started this thread :-)

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

Just a quick update, with blocking serial code I can log 72 bytes every cycle with the cycle time increased to around 6500 µs ... that seems acceptable for some test runs with adjusted PIDs.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

LOL, Cycle time was not an issue with version 1.7 when I used it. Wow, some folks just like to talk.....

With the SPI version, you will need to free up two of the motor pins as STATED earlier.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

P.S. Until there is an ESC that can provide the Process Loop Real Time RPM data, all of this logging, filtering etc is just beating a dead horse. Just my two cents.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

I like to talk ;-) SPI is an option on boards other then the 328P. And there are ESC which report RPM data back (https://www.mikrocontroller.com/index.p ... cts_id=504 and http://mikrokopter.de/ucwiki/BL-Ctrl_2. ... r_Drehzahl , German website), but you could also just measure the output frequency on one of the 3 motor wires ...

Logging exactly what is being measured helps to find algorithms which are faster/better. MultiWii works great in acro mode, but there are some quirks with level mode and altitude hold (at least with my copter). Having better ESCs and motors that react faster is a different problem.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

Level mode works perfectly for me, can't speak for anyone else.

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Inflight data logging HW+SW

Post by Hamburger »

like I said before
Hamburger wrote:We can agree on it being possible to design theoretical constellations.
I asked for real world data and derived new findings.

Sebbi wrote:
It is not theoretical. With all the vibrations going on the measurements aren't smooth at all ... one might not miss much when not logging every cycle and in many cases it might be enough to use this data, but I doubt that it will work for PID debugging. Real world data and derived new findings? I didn't have real world data so I simulated it ... I found that the I term in the getEstimatedAltitude() function needs very high I values to have an effect at all and the D term will cause oscillations when too high and finding optimal values for that might need 0.1 changes, but the GUI wont allow that. I want to check that with real world data (OpenLog arrived today *g*) ...

so that is an example for the use of simulator, not real world big data.
Also, the small angle approximation in the rotateV() function might cause problems. Checking if the deltas are really that small and the rotation has the same results as by using a real rotation matrix would be interesting ... the accelerometer is likely correcting that problem fast enough, but still ... I tried a different way to rotate the gravity vector and noticed that the gyro scale factor is way off too ... these are my theoretical findings so far. Needs accurate logging (or simulation) to confirm ;-)

If you think the algorithm being sub-optimal (for whatever reason), then you can implement another algorithm and test whether it gives truly better results. In the end this is the only thing that counts. You can do that without big data. Real world flight experience will be the decisive factor in the end.
Sebbi wrote:I like to talk ;-)

ok, got me. I took it too seriously :) I like to fly.
Nevertheless, in good spirit, please proceed, do all the logging you can do, dig in that chunk of data and come up with another algorithm that gives better flight performance. Yes, it is possible. To me, real world flight performance is what counts. The only thing that counts. Until then, I am done talking on this.
Have fun, it is the hobby.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

I like to fly, analyse data, dream in code and eventually fly really, really stable in stable mode ;-)

If you think the algorithm being sub-optimal (for whatever reason), then you can implement another algorithm and test whether it gives truly better results. In the end this is the only thing that counts. You can do that without big data. Real world flight experience will be the decisive factor in the end.


I think that is not the best approach to the problem. I need to know what part doesn't work as expected in which situation to improve upon it ... simulation helps, but it isn't the holy grail. "Real world flight experience", as you wrote, is the key ... and how would one objectively measure that without detailed logging and video data of the flight? ;-)

User avatar
EOSBandi
Posts: 802
Joined: Sun Jun 19, 2011 11:32 am
Location: Budapest, Hungary
Contact:

Re: Inflight data logging HW+SW

Post by EOSBandi »

Toro480 wrote:Hi EOSbandi. Is it possible to use your code on the new CRIUS All In One (mega 2560)?
I would like to record data (angle and PID corrections for example) in flash with your method. Did you do some improvement since january?


Ok guys, here is the stuff
I just got my AIO Pro boards, it seems that it has a perfetly aligned holes for SPI port plus the two additional pins... I'll redesign the blob for the dataflash, so it could be plugged directly to the AIO (after soldering the pins in). The code is ready and working. Wingui will be able to download the logs and visualise them in the LogBrowser.

I'm planning to make a small batch of these modules, approx cost will be around 25usd/blob + PP (small batch PCB and the flash chip is quite expensive).

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

No wind + openlog = tons of data

I am currently analysing one big logfile I recorded this afternoon. 167000 datapoints (4600 µs cycle time) in a littlebit over 12 minutes ...

Findings so far:
  • Apparently the sum ppm converter i am using (or the tx itself?) does some averaging over the last 140 ms (seven 20ms frames until from old value to new value switch)
  • A throttle value of 1317 lets the copter hover in its current configuration, however the BaroPID variable oscillates a lot (+- 15 max) even if it looked like a pretty stable hovering from my point of view
  • Vibrations suck ... they are really bad on acc in x and y direction, z acceleration is kind of ok ... gyros are ok (MPU6050). Even with smoothing it is a miracle that x/y acc values are of use for anything
  • Z acceleration correlates with BaroAlt and should/could definetly be used for altitude hold
  • gyroZero is an integer value, but should probably be a float ... gyro values are slowly drifting because of this
  • Spreadsheet applications are far too slow for this amount of data :-)

nothing really new, i know

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

:o

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Inflight data logging HW+SW

Post by Hamburger »

what would be the cycle time on this copter without the logging rig?
Does it matter?

Toro480
Posts: 4
Joined: Wed Jul 18, 2012 11:41 am

Re: Inflight data logging HW+SW

Post by Toro480 »

Thanks EOSBandi for your answer.
I'll buy this board. Could you tell me more about the blob. What does it means and how does it works?
Which connector on the board will be used for the blob?

@Sebbi : I'm also interested by the increase of cycle time when you activate your log function.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

My logging code has two options
1) log every cycle and increase cycle time according to bytes logged each cycle (36 bytes seems to be the limit before it needs to wait for the buffer to clear) ... it is currently saving 54 bytes per cycle increasing the cycle time from 3000 µs to 4600 µs
2) log when the tx buffer is empty enough. This will skip some cycles but does not influence the cycle time

Regarding the recorded vibrations: the MPU6050 has a fifo buffer and can use oversampling and filters to get smoother values with a fixed frequency ... it think there is not much difference in flight performance when MultiWii has a 10 ms cycle time and lets the sensor do the averaging ... new rx commands are coming in at 50 Hz and I don't know if things really improve when the speed of the motors is adjusted faster than 100 times per second with the high inertia those rotors have and all the averaging/smoothing going on in the current code. Time for a MultiMPU project? The MPU6050 (and soon 9050) is a lot cheaper and more accurate than any Nintendo Wii sensor available ;-)

P.S.: The openlog firmware has to be modified to work with binary log data, because it will switch into command mode when the escape character occurs too often (https://github.com/phollands/OpenLog has the neccessary changes)

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Inflight data logging HW+SW

Post by Hamburger »

do you know the impact of that increased cycle time on sensor data ? is there any? does reading frequency influence sensor data (beyond the number of data)?

So now you are in for the interesting part:
with your findings, what are you going to change to get better flight experience?
Please keep us updated.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

Hamburger wrote:do you know the impact of that increased cycle time on sensor data ? is there any? does reading frequency influence sensor data (beyond the number of data)?

So now you are in for the interesting part:
with your findings, what are you going to change to get better flight experience?
Please keep us updated.


This thread really should be moved or renamed PID tuning. As stated before, unless the Process control has real-time data from the ESCs, it is really just a guessing game and based on pure Luck finding the right settings. No two copters will have the same numbers because of AUW, Motor KV, Props used and a host of other variables.

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Inflight data logging HW+SW

Post by Hamburger »

copterrichie wrote:This thread really should be moved or renamed PID tuning.

I hope it does not get reduced to just PID tuning.
The more we know about the innards of MWii, the more it becomes a miracle it is flying as good as it is now.
Sebbis's findings from data sampling are not overwhelmingly surprising/new really, but over all it substantiates previous findings and thoughts, which is a good thing. Yes, data is still incomplete, loads of parameters remain unknown. Nevertheless, lessons could be learned.

The interesting point is: what he will come up with to better the situation.
Remember: mk always insisted on high data rates between FC and ESC, pushing their i2c interface (?). Now with the better ESC firmware which removes in-ESC-smoothing, for MWii the regular ESCs achieve comparable flight results with their low-as-ever data rates. Surprise. Huge gain for us.

So I am really interested to see the improvements he is going to derive from this. In the end we could all benefit.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

I agree with you Hambuger but according to what I read, Sebbis is trying to create more stable flight condition. This in my opinion is based upon the PID and the formulas used in the calculations. If there was a way to make the PID settings DYNAMIC and self Adjusting, then we are TALKING.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

Ok ... in order of appearance ;-)

Impact of cycle time on sensor data:
I'd say a change from 3000 to 4600 µs isn't that big of a change. First the acc is read, then the gyro then comes the infamous annexCode() function (which also calls the log function) and it ends with getting another data point from the gyro, averaging both values to be used in the next cycle to rotate the gravity vector and so on ... i don't think a higher frequency (than 100-200 Hz) for reading sensor data is that helpful with all the vibrations going on. All the possible details get averaged anyway ;-)

Impact on flight bahaviour:
Yeah, it changes a little bit, because the PID algorithms depend on timing. It felt smoother with the increased loop time, but that could also have been the lack of wind which makes controlling a copter sooo much easier ;-)

What am I going to change:
As I wrote in the wishlist thread I am interested in how a fixed time cycle loop (say 100 Hz) and using more accurate calculation would affect stable flight modes like position hold, altitude hold and level mode. Acro mode works perfectly for me and is relativly easy to get the copter level with minimal corrections ... i wonder if knowing how reliable sensor data is and how good the calculations are (are the angles off when it drifts away or does it think it is level while it's not?) could improve those things. I understand that PID tuning can achieve a lot, but no amount of tuning can compensate false sensor data. The vibrations on acc x/y make it next to impossible to determine small angle changes, at least on my copter. Balancing props and motors can help to a certain amount. Gyros are being used to rotate the gravity vector, but at least for me the gyros are drifting because of a 0,5 bias error caused by the bias/zero value being an integer and the scale factor isn't correct either. Somehow it works, but it is a miracle to me as Hamburger noted ;-)

My ultimate goal would be to have a copter which really stays level, even in wind. ;-)

Realtime data from ESCs and guessing game:
I generally agree, but that is what PID controllers are for. However, the best PID controller can not compensate for a scenario where the copter thinks it is level or not ascending/descending when it's not ;-) Lag of motor control isn't an issue when I do it manually in gyro only mode ... so the code doing it should be able to do it as well or better or am I wrong here?

PID tuning and making it dynamic:
I have some thoughts about that. The quality of a PID setting can be measured (e.g. standard deviation of attitude or height over a fixed time) and could be used in some kind of self adjusting mechanic that tunes itself to PID values that lead to better results. Obviously this can only be used to tune stable conditions. It would kind of work like the current use of "constrain()" to prevent wild corrections only on the PID values itself instead of the results. Is this what you had in mind, copterrichie? If not, please enlighten us ... I like to talk ;-)

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

Well, yes to some degree but I was thinking more of an expert system (short of artificial Intelligences) ;)


If this, do that,
if that, do this. etc.

http://en.wikipedia.org/wiki/Expert_system

P.S. Of course, it all depends upon the quality of the sensor input.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Inflight data logging HW+SW

Post by Sebbi »

So something like "if altitude oscillates too much around the setpoint reduce D value"?

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Inflight data logging HW+SW

Post by copterrichie »

Sebbi wrote:So something like "if altitude oscillates too much around the setpoint reduce D value"?


yea, that is the general idea or log the value for the human to make the determination to start out with. As the knowledge base improves, it can make these decisions on its own.

Post Reply