Flight data recording

Post Reply
Christian
Posts: 7
Joined: Sat Mar 23, 2013 7:58 pm

Flight data recording

Post by Christian »

Hello there,

I am quite new with programing but I want to use my Multiwii board to record Flight data from my flights. So I wonder if it is a good idea to safe the attitude and some I2C data on the board's EPROM memory or better to safe it in an external EPROM or other.

Also I already tryed to read what's in the EPROM memory following the Arduino tutorials, but it seems that there is a problem using Arduino´s Serial library to print the data on the serial monitor. As soon as I put in loop any Serial comand the sketch wont work.

As I said, I am very new in this :oops:

Thanks for any help!!!

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Flight data recording

Post by Hamburger »

eeprom is only 1k or 2k, so very limited for collecting inflight data for analysis.
Some new boards come with more storage or sd-card socket but MWii has no interfacing code yet? If it is not too much data, you can send it over serial-wireless to another device and record data there.

for eeprom and serial handling in MWii, you can look at the PERMANENT_LOG implementation. Could get you started.

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

Hamburger wrote:Some new boards come with more storage or sd-card socket but MWii has no interfacing code yet?


Currently, I am thinking and testing to implement a sd-card logger. The access to the sd-card is quite slow, so it will not support a sample rate of 3ms.
My idea is to integrate the sd-card to an i2c-gps device, because there are nearly 200ms time between two gps frames at a frequency of 5Hz.

Unfortunately, I am still struggling since the sd-lib does not seem to be compatible with the twi code used.
Is there someone else who has already developed a solution?

Christian
Posts: 7
Joined: Sat Mar 23, 2013 7:58 pm

Re: Flight data recording

Post by Christian »

HEY! thanks for responds. Now I am traing to put into work a "self made" SD card logger through SD library with arduino uno. My idea is that once I figure out how it works, use the code with the multiwii. The thing is that I think that it uses SPI comunicacion and I dont know if I can make it work on the multiwii... :?

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

Christian wrote:"self made" SD card logger


My effort on the development of an I2C_SDCard has resulted as follows:

QuadBow wrote: I am still struggling since the sd-lib does not seem to be compatible with the twi code

Probably, this issue is related to a RAM overflow causing restarts.

So, it is not possible to integrate the SD-card to the existing I2C-GPS without reducing many features of I2C-GPS.

Furthermore, SD-card access requires a lot of time (12ms for"Hello world !"). Therefore, an integration into an existing flight controller is not feasible. The issue is that the SPI-library waits for the completion of the transfer without using interrupts. Thus, the flight controller waits a long time delaying the cycle loop too much and risking an unstable operation.

byte SPIClass::transfer(byte _data) {
SPDR = _data;
while (!(SPSR & _BV(SPIF)));
...}

One possibility would be rewriting the SPI-lib in a more sophisticated way... smart programmers are welcome ;-)

My approach is to use another promini for the implementation of the I2C_SDCard. I am implementing following features:
- i2c-interface compatible with fast mode (400,000 baud)
- variables: voltage1, voltage2, current, power, rotations_per_second. temperature1, temperature2, pitch, roll, yaw, capacity_battery, energy_battery
- gps-vars: fix2d, fix3d, num_sats, lon, lat, alt, dist_to_home, ground_speed, ground_course (no nav variables)
- optional: min-, mid-, and max-values (not recommended for high speed applications like flight controllers)
- commands: sd_init, sd_start, sd_restart, sd_stop, sd_final, set_sd_rate, sd_send, sd_new_data
- flexible recording time above 100ms
- sleep instruction for power saving between records
- future extention: lcd integration (if enough flash and Ram remains...)

Christian
Posts: 7
Joined: Sat Mar 23, 2013 7:58 pm

Re: Flight data recording

Post by Christian »

HUM.... I am just thinking, is it possible to use the mini-usb conection of the multiwii, and instead of transimitting to the pc, transmit to an normal SD cardlogger that you can buy in a shop?, would be something like pluging toghether the usb terminals and try serial comunication?

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

Christian wrote:is it possible to use the mini-usb conection of the multiwii, and instead of transimitting to the pc, transmit to an normal SD cardlogger that you can buy in a shop?


In principle yes, but:
Multiwii sends out a binary format via USART. written on a SD-card this binary information will not help you very much...
You had to rewrite the multiwii serial code and deviate from the multiwii path, which I do not recommend.

Christian
Posts: 7
Joined: Sat Mar 23, 2013 7:58 pm

Re: Flight data recording

Post by Christian »

Hello there, well after thinking about it a little more, i've been thinking, couldn't i just use the multiwii as sensor shild and transfer data through I2C to arduino uno card, and from there to SD card, that way the multiwii would work just as ussual and transfer just the yaw, pitch, bank angles and meaby altitud and that's it, it would just need little memory space for multiwii. So the thing is, can I use the multiwii like this? like I2C slave it would be.

And would then the data transfer to SD card be faster?

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

Christian wrote:Hello there, well after thinking about it a little more, i've been thinking, couldn't i just use the multiwii as sensor shild and transfer data through I2C to arduino uno card, and from there to SD card, that way the multiwii would work just as ussual and transfer just the yaw, pitch, bank angles and meaby altitud and that's it, it would just need little memory space for multiwii. So the thing is, can I use the multiwii like this? like I2C slave it would be.


That was actually what I proposed... However, I have also given more thought to this feature. The issue with I2C ist the required time, since the processor is awaiting every bit to be finished at the current multiwii implementation. In my view the better way would be to use serial communication - using the hardware serial via the multiwii configuration tool protocol without any further burden to the processor or additional code to be implemented in multiwii.

Thus, we need only a second arduino board (promini or uno) with SD-card-slot and a free hardware serial port and to implement the related software...

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Flight data recording

Post by copterrichie »

I personally have used the Sparkfun Openlog and it does work very well. I believe there is new firmware now that allows it to record binary, at the time I was using it, data had to be in ASCII. https://www.sparkfun.com/products/9530

Christian
Posts: 7
Joined: Sat Mar 23, 2013 7:58 pm

Re: Flight data recording

Post by Christian »

copterrichie: thanks for the link, probably will buy it. It uses SPI comunication right?

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Flight data recording

Post by copterrichie »

Christian wrote:copterrichie: thanks for the link, probably will buy it. It uses SPI comunication right?


No, it is serial.

Christian
Posts: 7
Joined: Sat Mar 23, 2013 7:58 pm

Re: Flight data recording

Post by Christian »

Thanks alot QuadBow, just quick very stupid question: so I probably will need and USB conector that conects the USBmini port of multiwii with the usb port of arduino uno right?, and is it going to work alright suppling 5V through Rx conection for example.. or servo conection... ?

again, i am very new at this... so probably very stupid question...

Christian
Posts: 7
Joined: Sat Mar 23, 2013 7:58 pm

Re: Flight data recording

Post by Christian »

okssss.. so i need to read ab serial comms.... thanks alot!!!! i have it shipping from sparfunk so hopefully when it arrives I will have multiwii running with arduino!

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

copterrichie wrote:https://www.sparkfun.com/products/9530


Interesting link,

copterrichie wrote:No, it is serial.


but, the communication to the SD-card seems to be done via SPI,
whereas the communication to the multiwii-master uses serial.

copterrichie wrote:I believe there is new firmware now that allows it to record binary, at the time I was using it, data had to be in ASCII.

I have found in the code only ASCII elements. Additionally, binaríes are not described in https://github.com/sparkfun/OpenLog/wiki/Command-Set .
Dealing with binaries requires the agreement to a protocol. (multiwiiconf?)

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Flight data recording

Post by copterrichie »

I would assume using the Multiwii protocol, it could be made to handle binary but I never attempted it. I simply recorded everything in ASCII comma delimited, then read the date with Excel.

Quad FRITZ
Posts: 44
Joined: Wed Apr 25, 2012 11:21 pm

Re: Flight data recording

Post by Quad FRITZ »

This board seems to have lots of on board flash memory: http://www.rctimer.com/index.php?gOo=go ... oodsid=765
Perhaps this might help?

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

Quad FRITZ wrote:This board seems to have lots of on board flash memory: http://www.rctimer.com/index.php?gOo=go ... oodsid=765
Perhaps this might help?


This board is based upon an Atmega2560 with 256 kB flash and 4 kB eeprom.
For data recording the eeprom size is important, not the flash size.
Writing every 100ms 100 bytes onto the eeprom means is will be full after 4 seconds.
Finally, how do you want to get the data out of the eeprom?

The best solution is an additional board with sd-card-slot.
After recording (and it takes some time to fill for example the 4 GB of the card) you just take the sd-card out and plug it into the sd-card-reader of your pc or notebook and you can directly access the data e.g. in Excel format and work further on it.

TextZombie
Posts: 24
Joined: Mon Nov 14, 2011 2:05 pm
Location: Sheffield, UK
Contact:

Re: Flight data recording

Post by TextZombie »

I think it's more that the AIO Pro 2 has an Atmel 45DB161E 16Mbit DataFlash on it.
MegaPirateNG has a library to access it, I imagine though it's too niche to spend time trying to integrate it.

P

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

TextZombie wrote:I think it's more that the AIO Pro 2 has an Atmel 45DB161E 16Mbit DataFlash on it.


That would be an explanation for mentioning it.
However, this is not mentioned att all, neither in the link provided above, nor in hobbyking's description of the board http://www.hobbyking.com/hobbyking/stor ... _V2_0.html .
Assuming there were a dataflash on it, it would be interesting to know, where the related software could be found.
As to my knowledge, it is not integrated in the multiwii software.

User avatar
EOSBandi
Posts: 802
Joined: Sun Jun 19, 2011 11:32 am
Location: Budapest, Hungary
Contact:

Flight data recording

Post by EOSBandi »

integration of the dataflash was done, i have to update it to the latest code and remove c class libraries from it, since those are not meet with the current coding standards of a Multiwii. In WinGui there is even a first version of the log downloader and analyzer.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Flight data recording

Post by Sebbi »

Just a suggestion, as logging is quite similar to telemetry transmissions could we have a unified system to either transmit data via downlink or log it to an SD-card / dataflash? Could be integrated with the current serial communication as some telemetry (the GUI is certainly one form of it) also reacts to commands ...

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Flight data recording

Post by Hamburger »

Sebbi wrote:Just a suggestion, as logging is quite similar to telemetry transmissions could we have a unified system to either transmit data via downlink or log it to an SD-card / dataflash? Could be integrated with the current serial communication as some telemetry (the GUI is certainly one form of it) also reacts to commands ...

In real life, logging and telemetry address different issues and meet different, sometimes contradicting requirements.
For example, logging has to be fast (enough) so you do not loose any (of the massive) data, the format is mostly irrelevant (so long as you can decode it later). On the other hand, telemetry is about presentation, so the format must be tailored to suit this aspect, the speed only has to suffice the viewer, some lost data should not interfere with the overall accuracy of the presentation.

If there is a benefit to be had from bringing parts of logging and telemetry together, then it will be good.

User avatar
EOSBandi
Posts: 802
Joined: Sun Jun 19, 2011 11:32 am
Location: Budapest, Hungary
Contact:

Re: Flight data recording

Post by EOSBandi »

Sebbi wrote:Just a suggestion, as logging is quite similar to telemetry transmissions could we have a unified system to either transmit data via downlink or log it to an SD-card / dataflash? Could be integrated with the current serial communication as some telemetry (the GUI is certainly one form of it) also reacts to commands ...

It is all about speed. SD card is definitely a no go. It has serious speed and lag issues...
Telemetry is for remote sensing, datalog is for logging. With a Dataflash you can log at 50Hz without any significant impact on the cycletime. With the serial telemetry it is impossible. So i don't think that merging the two is a good idea. Also with datalog you can log theoretically ANY internal variable....

PatrikE
Posts: 1976
Joined: Tue Apr 12, 2011 6:35 pm
Location: Sweden
Contact:

Re: Flight data recording

Post by PatrikE »

Maby a Modification of the GhettoStation to reqest data from MWii and write to a SD-card.
viewtopic.php?f=8&t=3350


Modifications needed
- Implement MSP comunication.
- SD-card support.

TextZombie
Posts: 24
Joined: Mon Nov 14, 2011 2:05 pm
Location: Sheffield, UK
Contact:

Re: Flight data recording

Post by TextZombie »

QuadBow wrote:However, this is not mentioned att all, neither in the link provided above, nor in hobbyking's description of the board

The hobbyking board isn't the same I think, the DataFlash only got added to the Crius in v2 and the hobbyking board looks to be a copy of a v1. The RCtimer page says "Onboard 16Mbit Dataflash chip for automatic datalogging" which had me go look at mine with a magnifier to find it. Documentation on the Crius stuff is fairly appalling, but at least they don't scratch the chip numbers off.

P

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

TextZombie wrote:The hobbyking board isn't the same I think, the DataFlash only got added to the Crius in v2 and the hobbyking board looks to be a copy of a v1. The RCtimer page says "Onboard 16Mbit Dataflash chip for automatic datalogging" which had me go look at mine with a magnifier to find it. Documentation on the Crius stuff is fairly appalling, but at least they don't scratch the chip numbers off.


Thank you for the clarification.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Flight data recording

Post by copterrichie »

Personally, I believe a Class 10 SD card will handle anything we can possibly try to write to it. The problem exist of how to connect it to the flight controller.

User avatar
NikTheGreek
Posts: 348
Joined: Thu Dec 08, 2011 4:17 pm
Location: Greece
Contact:

Re: Flight data recording

Post by NikTheGreek »

copterrichie wrote:Personally, I believe a Class 10 SD card will handle anything we can possibly try to write to it. The problem exist of how to connect it to the flight controller.


This SD cards are very cheep http://www.ebay.com/itm/New-1pcs-SD-Car ... 5d3d4f9271.
but must be connected via SPI .
Do you think that can be implemented in conjuction with the I2C GPS in the same board ?

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Flight data recording

Post by copterrichie »

NikTheGreek wrote:
This SD cards are very cheep http://www.ebay.com/itm/New-1pcs-SD-Car ... 5d3d4f9271.
but must be connected via SPI .
Do you think that can be implemented in conjuction with the I2C GPS in the same board ?


I think it would be possible but now the issue becomes, how much traffic can the I2C buss handle?

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

NikTheGreek wrote:Do you think that can be implemented in conjuction with the I2C GPS in the same board ?


I tried already and ran into ram limitations. Look at my post on the first page.
So, copterrichies' proposal https://www.sparkfun.com/products/9530 would be the best idea if you want to use a sd-card.

User avatar
EOSBandi
Posts: 802
Joined: Sun Jun 19, 2011 11:32 am
Location: Budapest, Hungary
Contact:

Re: Flight data recording

Post by EOSBandi »

NikTheGreek wrote:
copterrichie wrote:Personally, I believe a Class 10 SD card will handle anything we can possibly try to write to it. The problem exist of how to connect it to the flight controller.


This SD cards are very cheep http://www.ebay.com/itm/New-1pcs-SD-Car ... 5d3d4f9271.
but must be connected via SPI .
Do you think that can be implemented in conjuction with the I2C GPS in the same board ?

And what for ? the serial port of the I2C-GPS is occupied by the GPS... so no connection to the flight controller.
Using an SD card (with FAT filesystem) is a memory hog, and without interrupts in SPI mode is definitely has blocking calls. And on a 16Mhz arduino you got max 8Mhz SPI speed....
So I still not seeing a viable usage for SD card onboard... unless it is used only for logging GPS position, and driven by a separate processor.... But it is definitely overkill, compared to a dataflash memory...

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Flight data recording

Post by copterrichie »

The SD card I assume would connect to the SPI port on the I2C-GPS board.

User avatar
EOSBandi
Posts: 802
Joined: Sun Jun 19, 2011 11:32 am
Location: Budapest, Hungary
Contact:

Re: Flight data recording

Post by EOSBandi »

copterrichie wrote:The SD card I assume would connect to the SPI port on the I2C-GPS board.

Yeah, but then it should log only the gps position data. IF the sd card handler fit's into the memory (not flash, but RAM).

QuadBow
Posts: 532
Joined: Fri Jan 04, 2013 10:06 am

Re: Flight data recording

Post by QuadBow »

copterrichie wrote:The SD card I assume would connect to the SPI port on the I2C-GPS board.


QuadBow wrote:I tried already and ran into ram limitations. Look at my post on the first page.

Sebbi
Posts: 478
Joined: Sun Jul 08, 2012 1:08 am
Location: Germany
Contact:

Re: Flight data recording

Post by Sebbi »

EOSBandi wrote:
Sebbi wrote:Just a suggestion, as logging is quite similar to telemetry transmissions could we have a unified system to either transmit data via downlink or log it to an SD-card / dataflash? Could be integrated with the current serial communication as some telemetry (the GUI is certainly one form of it) also reacts to commands ...

It is all about speed. SD card is definitely a no go. It has serious speed and lag issues...
Telemetry is for remote sensing, datalog is for logging. With a Dataflash you can log at 50Hz without any significant impact on the cycletime. With the serial telemetry it is impossible. So i don't think that merging the two is a good idea. Also with datalog you can log theoretically ANY internal variable....


Well, I used an openlog to log flight data. It all depends on how much data you want to log. At a baudrate of 115200 (non blocking, just like the GUI) it can log around 11 kb of data per second or when logging every cycle it can write approximately 38 bytes per cycle. That's not a lot, but even this "slow" rate would fill up a 16 mbit dataflash in only 3 minutes.

If you want to log whole flights SD cards are perfect, because gigabytes! Openlog is a great example how to do it with just a 328P as coprocessor (the thing is tiny!).

And even with different speed it is still the same process writing something to a log or to a transmitter. I'm just suggesting that it you be a unified system, e.g. one call to do_telemetry() and this function does whatever it has to do according to compile flags (similar to the current LCD telemetry and serial communication implementation).

Post Reply