Reorganising source *.c code with *.h headers; Eclipse IDE

Hamburger · Post by **Hamburger** » Sun Feb 03, 2013 7:52 pm

no big difference for the in-experienced in using the IDE or another program to configure required set of modules.
Obvious difference is that user friendly program does not exist yet.

copterrichie · Post by **copterrichie** » Sun Feb 03, 2013 8:09 pm

It could also be a library. Hmmm

alll · Post by **alll** » Mon Feb 04, 2013 12:15 am

Don't cry victory too soon...

I tried to upload the hex to the board, and it doesn't work in the gui, straight line...

Even if i compile in arduino ide, and upload, same thing :evil:

manu
PS:as said before, it was a quick and dirty try :roll:

copterrichie wrote:Eureka!

Huston, we have liftoff.

Thank you.

alll · Post by **alll** » Wed Feb 06, 2013 11:54 pm

EUREKA!

I could not compile in Eclipse the inline SerialOpen function, in Adruino IDE i got no problem. So removing the inline directive made it...??? Is there a compiler option to set?
//void inline SerialOpen(uint8_t port, uint32_t baud) {
void SerialOpen(uint8_t port, uint32_t baud) {

alll · Post by **alll** » Wed Feb 06, 2013 11:57 pm

copterrichie · Post by **copterrichie** » Wed Feb 06, 2013 11:59 pm

Congratulations! Let me see if I can repeat what you have done.

copterrichie · Post by **copterrichie** » Thu Feb 07, 2013 12:13 am

I am still getting nothing, will have a closer look later.

Alexinparis · Post by **Alexinparis** » Thu Mar 14, 2013 3:22 pm

Hi,

As we are going on this way for the next release, I read a lot on this subject yesterday.

I think the way files are handle in your zip (alll) does not address fully the .c .h approach.
According to http://arduino.cc/en/Hacking/BuildProcess
every file with no extension (ie .ino files) are concatenated, a prototype is generated, and the big file is then compiled.

The proper way would be to use independent .c or .cpp so that they are compiled independently in .o.

So to my mind, the current file separation is good and should be adapted this way in a flat repositiory:
Alarms.ino -> replaced by Alarms.cpp + Alarms.h
EEPROM.ino -> replaced by EEPROM.cpp + EEPROM.h
GPS.ino -> replaced by GPS.cpp + GPS.h
IMU.ino -> replaced by IMU.cpp + IMU.h
LCD.ino -> replaced by LCD.cpp + LCD.h
MultiWii.ino -> we keep MultiWii.ino + MultiWii.h
Output.ino -> replaced by Output.cpp + Output.h
RX.ino -> replaced by RX.cpp + RX.h
Sensors.ino -> replaced by Sensors.cpp + Sensors.h
Serial.ino -> replaced by Serial.cpp + Serial.h
config.h -> same
def.h -> same
tinygps.h -> same

.cpp or .c : I don't know. All I know is Arduino interpret the .ino as a C++ file. The most convenient extension to avoid things like extern "C" {} seems to be .cpp

Of course adapting some c-style syntax to be friendly with other compiler/IDE is an option, but it must compiled as before with no customization in the Arduino IDE.

So, another zip would be welcomed

alll wrote:Hi,

Here the sources and headers (version21). I use the Eclipse Arduino plugin.
I managed to compile for #define COPTERTEST 1,2 and 3. For #define COPTERTEST 4 (GPS) i didn't (a big beast)!
I really think it is time to reorganize the mwii code and make more independent modules / functions. While doing this exercise, you see that references goes into all directions of the modules. Also the extensive usage of #defines / sub defines, to try to stay generic, doesn't ease the maintenance and readability.
There are too many dependencies imho. Sorry to be so negative, but i am still amazed how well mwii flies! A good reason to continue adding / improving the functionalities.

It would be nice to have this sort of structure:

Input data:
Reading/sampling raw rx,sensor,GPS,... data
Normalizing the input data: zero, scaling, ...
Transforming : mixing, PID, ...

Output data;
PWM, serial, ...

manu

alll · Post by **alll** » Tue Jun 11, 2013 6:39 pm

Hi Alex,
I am back on this topic, is it still ongoing?

I found a rather easy way to make changes to the mwii arduino ino file organisation to be able to use the code in Eclipse with the Arduino Eclipse plugin (at least the indexer is happy with it

:

now you will see a lot of red indexer errors, to help him finding all references the basic idea is :

Code: Select all

#include "Arduino.h"
#include "config.h"
#include "def.h"
#include "globalv.h"

Code: Select all

#ifndef DEF_H_
#define DEF_H_
...
#endif /*DEF_H_*/

thats it, only problem still have is with the serial library "HardwareSerial.cpp" with UCSR0B, ...
There is a comment in the cpp file:

Code: Select all

// this next line disables the entire HardwareSerial.cpp, 
// this is so I can support Attiny series and any other chip without a uart
#if defined(UBRRH) || defined(UBRR0H) || defined(UBRR1H) || defined(UBRR2H) || defined(UBRR3H)

all after #if defined(...
is ignored?!

manu

clough42 · Post by **clough42** » Thu Jun 13, 2013 5:21 pm

alll wrote:I am back on this topic, is it still ongoing?

I'm also interested in this effort.

I reorganized the MultiWii 2.1 code into .cpp/.h files late last year to see if it could be done. I ultimately got it working in Eclipse, but it didn't work well in the Arduino IDE, and the changes were such that it wasn't going to be possible for me to take ongoing changes from the repository or submit anything back and expect it to merge, so I abandoned it. I'm not really interested in hand-merging my own private copy of the code forever.

Alex said, "We are going this way in the next release." Is this effort in a repository somewhere? I.e. is there a branch I can check out?

I'd be happy to help if I can, but I realize I'm new here and I don't want to step on anyone's toes.

Alexinparis · Post by **Alexinparis** » Thu Jun 13, 2013 10:01 pm

alll wrote:Hi Alex,
I am back on this topic, is it still ongoing?

Hi,

Yes, it's still ongoing, but in sleeping mode

If it's just some file reorganization to be compliant with eclipse, I'm not interested
I'm interested in a proper way:
- c/h or C++/h so that things are compiled independently in .o
- compatible with current Arduino IDE with no mod
- compatibility with other IDE like eclipse is a bonus

So far, I didn't manage to make a draft.
I would like to see an example based on the current _shared.

timecop · Post by **timecop** » Fri Jun 14, 2013 1:16 am

To make it proper, the changes would have to be substantial. I don't think this is something one would do just to verify if concept is OK or not. Personally I'd suggest making a small test project with maybe one cleaned up driver (mpu) and no ifdefs /lcd/etc and make it work with tarduino ide requirement, then go from there.

alex.khoroshko · Post by **alex.khoroshko** » Fri Jun 14, 2013 3:33 pm

Hello!
I tried working in arduino IDE and it's such a pain. Eclipse idea is good, but it's still stuck with the bootloader. So I was always interested - why not just flash the good-old way, via USB ASP? Everyone can use at least AVR studio, which is way better, than arduino IDE.
As for files, it is common to divide into plenty of small files. For example, I would create separate file pid.c and pid.h. Create the data structure, used by pid (input, output, integrators, settings etc). So now, one code is being used for all pids. for example like:

Code: Select all

pAcroRoll_PID_def->input = someVar;
PID (pAcroRoll_PID_def);
anotherVar = pAcroRoll_PID_def->output;

pAcroPitch_PID_def->input = someVar2;
PID (pAcroPitch_PID_def);
anotherVar2 = pAcroPitch_PID_def->output;

Amagine, how much space that would save - one chunk of code for all the pids in the quad. This code is very easy to read (all the subcode is in pid.c, not main loop, for example), and also this implementation is very fast, because X Y Z registers are used for indirect addressing, and all data for PID is placed side-by-side in one RAM area. I have no idea, how to implemend that visually enough in current file structure.

Also, a bit offtopic - direct compile for the target hardware has plenty of options. I worked a lot with Cortex M3 and M4 processors and Eclipse IDE - direct compile / upload is awesome. For example, atmel Cortex M3 has USB driver - it can upload firmware by the software, provided by atmel (no programmer needed, just USB). Not only that, I have almost ready to use USB-ready bootloader for Atmel, Ti stellaris and STM32 cortex M4 processors. When the bootloader is there (this might be a problem for some platforms), you connect the board to PC, it detects as flash drive, copy the firmware.bin file, push the button - it is flashed (I used it for other tasks,it'll work this way as well). There should be enough RAM to accept the firmware though.

Hamburger · Post by **Hamburger** » Fri Jun 14, 2013 4:32 pm

We can and do use eclipse already. There are several makefile environments which support arduino style compile and upload.

In your argumentation, please do not mix coding style and file layout&IDE - you can already use whatever data structure is appropriate - nothing to do with files.

alll · Post by **alll** » Fri Jun 14, 2013 6:02 pm

Hi,

Here a first attempt to get multiwii (i took ver2.2) compiled from within Eclipse with the help of the Arduino Eclipse plugin.
The "MultiWii22" directory compiles still with the Arduino IDE (i tried all the 5 #define coptertests)
The Eclipse CDT indexer is happy with this structure (already a good start

), however the build compiler/linker is not!
There are still "red bugs" when functions are "forward declared", i suppose this has something to do with the 1 pass indexer analyses?
If a more knowledge person can bring it one step further would be wonderful!

All i did:

Code: Select all

#include "Arduino.h"
#include "config.h"
#include "def.h"
#include "MultiWii.h"

added to all xxx.h files

Code: Select all

#ifndef xxx_H_
#define xxx_H_
...
#endif // xxx_H_

Follow these instructions to get your Arduino projects done (upload via avrdude included) from within Eclipse
http://www.baeyens.it/eclipse/Install.html
in preferences asociate *.ino to cpp an c source files

In Eclipse:

Any suggestions are welcome.

I can say that getting the indexer parse the coditional compile directives makes the code so more readable, and i don't talk about the references search ... very helpfull.
Once i get it compile from within Eclipse, i could make eventual (modest) contributions with ideas i want to try out.

manu
PS: if a lot of people are already using Eclipse, could one share how they do it "STEP by STEP", thanks

casquad · Post by **casquad** » Fri Jun 14, 2013 6:30 pm

Hello everybody,

I am new to this project and I have not found a specific forum to introduce myself to the rest of the team, so let me do it here. I have worked in embedded systems for the last five years, mainly for wireless sensor and automotive applications. I would like to build my own quadcopter in my free time and after googling some state-of-art open source flight control software, I decided to go for MultiWii. I have great experience in ARM and PPC platforms, but I have never worked with arduino-based platforms. As usual, the first thing I did was installing the tooling and downloading the source code from svn.

I must say that working with Arduino IDE is really painful (at least for a project of this size). I have always developed my code using eclipse so I installed the AVR plugin and manage to compile the MW project (not so hard once I saw what Arduino IDE was really doing in the background). During the job, I saw this thread and found it really interesting.

Keeping the source simple so it can be compiled easily in Arduino IDE is really helpful, but I agree with other developers here that given the current size of the project (and it looks it will keep growing), a new layout based on individual .c/.h files should be proposed. Is there any real planning to go into this direction? I would be glad to cooperate if necessary.

Regards

clough42 · Post by **clough42** » Fri Jun 14, 2013 7:02 pm

alll wrote:The Eclipse CDT indexer is happy with this structure (already a good start ), however the build compiler/linker is not!
There are still "red bugs" when functions are "forward declared", i suppose this has something to do with the 1 pass indexer analyses?
If a more knowledge person can bring it one step further would be wonderful!

I will take a crack at it tonight. I'd really like to see separate .c/.cpp and .h files for each of the existing .ino files, with proper forward declarations internal to each .c/.cpp file and only the external interfaces exposed in the .h files. I'm also imagining a separate globals.h file with extern declarations for the globals in multiwii.c/cpp.

It looks like things are fairly well organized already, so it shouldn't be too much of a slog, but it's tough to tell how much work is really involved without digging in and doing it.

The Arduino development environment makes it really easy to get something going quickly without a lot of software engineering experience. The automatic prototype generation makes it so you don't have to worry about forward declarations, but the downside is that you don't get any help keeping things encapsulated. It's really easy to cross-couple things without making an intentional decision to do so. This is probably off-topic. I'll take a look at the code tonight and report back.

casquad · Post by **casquad** » Fri Jun 14, 2013 8:27 pm

I have taken a look at the code and I think we can modularize it progressively without breaking the Arduino IDE compilation. I would suggest following the next steps:

1) Extract all relevant type definitions into a common header file "types.h". There are currently variables that define the structure type in the declaration itself. This is a bad idea for modularization.
2) Create a header file for each module (eeprom.h, gps.h, imu.h, lcd.h, etc) containing variable and function declarations that are used outside the module
3) Include relevant header files in each .ino file (config.h, def.h, etc)

Of course, to avoid multiple inclusion all header files must be properly protected by "#ifndef _xxx_ #define _xxx_ ...".

But since I am new to the project, maybe I am missing something. Please let me know any comments.

alll · Post by **alll** » Fri Jun 14, 2013 8:48 pm

One of the "absolute" requirements is that all files are in a folder, you open the main ino file in the Arduino IDE, compile and upload.
And i don't know (maybe i understood it wrong) why they want to keep a absolute minimum of files...?

It is really nice to immediately see which functions/variable/structures are exposed and which are local.

Now with Eclipse like CDT code indexers, even if the code is not following the "common c way", you can still fairly well navigate and understand the logic better.

Come on, let's make it compile in both IDE's, so the moderaters will be convinced to follow.

cross my fingers,
manu

casquad · Post by **casquad** » Sat Jun 15, 2013 10:29 am

I have attached the MW project separated into cpp and h files. It is compiling both in Eclipse and Aduino IDE, so we can enjoy the wonderfull advantages of Eclipse: code navigation, sintax coloring, etc. I have only tested one configuration, but it is only a matter of resolving the dependencies between files and moving missing declarations to .h files.

Btw, I have checked the svn and branches are not real branches, but direct commits into the "branches" folder (the only real branch is DaedalusFC). This is completely against the principles of svn. How do you do the merges between branches or track the history of a file? Is there any responsible for the svn management? I have not tried to commit anything yet, do I need permissions to do it?

alll · Post by **alll** » Sat Jun 15, 2013 11:07 am

1000x thanks for this!!!!! And really happy to have a "professional" on board to guide us for the coding "best practices". Everybody, i am sure will benefit from it.

So we are a big step further ("we" : the ones that could only use the "crappy Arduino IDE"...) . I really hope the mwii "owners" take the afford now to make it true, and official asap. Most of the work is done, and proved to work.

All thanks,
manu

alll · Post by **alll** » Sat Jun 15, 2013 12:07 pm

tested it out, its working, had to add some forward declares in sensors.
I have the whole Eclipse Arduino AVR toolchain working (upload via avrdude included!)

And it compiles still from within Arduino IDE

No exuses anymore to go further with it

casquad · Post by **casquad** » Sat Jun 15, 2013 12:23 pm

I am glad to hear that :-)

I would like to make a step-by-step tutorial about how to integrate the MW project and arduino libraries in eclipse. It is not complex, but a bit tedious. I will upload it when I have it finished.

I am waiting for my board (I ordered it yesterday from Hobbyking), so at this moment the only test I can do is compiling. I do not know if there are side-efects on runtime, but I will be able to check it soon.

Who should take the decission of migrating the project to cpp/h files and perform the necessary testing?

timecop · Post by **timecop** » Sat Jun 15, 2013 12:28 pm

casquad wrote:Btw, I have checked the svn and branches are not real branches, but direct commits into the "branches" folder (the only real branch is DaedalusFC). This is completely against the principles of svn. How do you do the merges between branches or track the history of a file? Is there any responsible for the svn management? I have not tried to commit anything yet, do I need permissions to do it?

hahaha..
lurkmoar, you'll learn that even uploading zip files into svn is ok.

alll · Post by **alll** » Sat Jun 15, 2013 1:06 pm

Try to stay positive, nobody is perfect, except ...

casquad · Post by **casquad** » Sat Jun 15, 2013 1:12 pm

Sorry, I was just trying to get familiar with the project and one of the most important aspects is the svn management. I was thinking about creating a new branch for cpp/h migration without affecting the trunk development, and merge the migration into the trunk at some point in the future. I did not mean to offend anybody.

timecop · Post by **timecop** » Sat Jun 15, 2013 1:26 pm

No offense meant, sorry if it sounded that way.
But yes, SVN is not used like that here.
Basically you'd replace MultiWii_shared with C version and go from there.
But it's too major of a change, so it would need others approval.

casquad · Post by **casquad** » Sat Jun 15, 2013 1:58 pm

No problem at all :-)

English is not my native language, and sometimes happens that I missunderstand some expressions. I apologize.

As I said, I would suggest creating a new branch "branches/CppMigration" from "trunk/MultiWii_shared" and use this branch for our "crazy" idea of migrating MW to cpp/h files. After our job is finished, approvals will be able to take the decission of merge this branch into the trunk or keep both development lines open so each developer may choose whatever he wants (classical Arduino IDE with ino files or Eclipse with cpp files).

If the branch is properly created, it should be easy to merge improvements and new features from trunk to branch and viceversa. We can create the branch with a simple command:

svn copy http://multiwii.googlecode.com/svn/trunk http://multiwii.googlecode.com/svn/bran ... pMigration -m "Created new branch for cpp/h migration"

I can commit my code with Alll modifications to this new branch, so other developers can contribute to develop the C code without breaking the trunk. I would be glad to cooperate if project managers like this way.

alll · Post by **alll** » Sun Jun 16, 2013 6:59 am

To keep track of all these boards and features

Code: Select all

/*************************************************************************************************/
/****           config.h : CONFIG files selector                                                      ****/
/*************************************************************************************************/
#ifndef CONFIG_H_
#define CONFIG_H_

#include "CRIUS_LITE_test301205.h"
//#include "CriusSEi2cGPS_v01.h"
//#include "FREEIMUv043_v03.h"

#endif /* CONFIG_H_ */

manu

Alexinparis · Post by **Alexinparis** » Sun Jun 16, 2013 12:57 pm

Hi,

It seems you managed to do it

There are apparently still some tweaks to add, but the principle is exactly what I have is mind.

So to go further:

1) You can create a branch or folder (pm me with a valid email for googlecode access, because yes you need a permission for it). But the counterpart is you will have to update this branch regularly with _shared evolution until everything is fine. Once it's ok, _shared will be replaced by this branch.

2) If not too long, we can decide to freeze _shared evolution for some days until you finalize the full port work. Once done, we just have to replace the _shared folder and continue with this.

Things should at least work with all coptertest variant to think about a switch.
This page is a good indicator for configs: http://sebbi.de/multiwii/

casquad wrote:I have attached the MW project separated into cpp and h files. It is compiling both in Eclipse and Aduino IDE, so we can enjoy the wonderfull advantages of Eclipse: code navigation, sintax coloring, etc. I have only tested one configuration, but it is only a matter of resolving the dependencies between files and moving missing declarations to .h files.

Btw, I have checked the svn and branches are not real branches, but direct commits into the "branches" folder (the only real branch is DaedalusFC). This is completely against the principles of svn. How do you do the merges between branches or track the history of a file? Is there any responsible for the svn management? I have not tried to commit anything yet, do I need permissions to do it?

Hamburger · Post by **Hamburger** » Sun Jun 16, 2013 5:30 pm

Please verify output code sizes are same as within arduino ide or smaller.

alll · Post by **alll** » Sun Jun 16, 2013 8:00 pm

Hamburger wrote:Please verify output code sizes are same as within arduino ide or smaller.

The same

ECLIPSE:

Code: Select all

20:55:01 **** Incremental Build of configuration Release for project MultiWii ****
make all 
Invoking: Print Size
avr-size --format=avr --mcu=atmega328p MultiWii.elf
AVR Memory Usage
----------------
Device: atmega328p
Program:   15438 bytes (47.1% Full)
(.text + .data + .bootloader)
Data:        987 bytes (48.2% Full)
(.data + .bss + .noinit)
Finished building: sizedummy
20:55:01 Build Finished (took 134ms)

ARDUINO

Code: Select all

d:\Temp\AppData\build7539297584309471069.tmp\MultiWii.cpp.hex 
Binary sketch size: 15,438 bytes (of a 32,256 byte maximum)

casquad · Post by **casquad** » Sun Jun 16, 2013 8:55 pm

As soon as I have permissions in the svn, I will create a branch for cpp/h development and commit my first version. This cpp code will be the start point for the migration so everybody can add their contributions to this new branch.

I will let you know when this is done.

timecop · Post by **timecop** » Mon Jun 17, 2013 12:28 am

Try using a newer avr-gcc than whats included with tarduino IDE - previous attempts at C-ifying the code have consistently produced smaller binaries. Also might want to try doing -Os or -O3 or similar for optimization options.

alll · Post by **alll** » Mon Jun 17, 2013 6:15 am

Yes, all avr compiler + avrdude are included in the Arduino IDE.
All that is needed (follow install instructions i mentioned abobe) :
*Install Java JRE,
*Install latests version of Arduino,
*Install Eclipse C+C++,
*Install Arduino Plugin,
And you should have your toolchain working.
I am happy this is finally on the road, no way back!
manu
PS: could a similar setup (using Eclipse) be made/explained for a STM or other mcu toolchains?

EOSBandi · Post by **EOSBandi** » Mon Jun 17, 2013 8:56 am

alll wrote:Yes, all avr compiler + avrdude are included in the Arduino IDE.
All that is needed (follow install instructions i mentioned abobe) :
*Install Java JRE,
*Install latests version of Arduino,
*Install Eclipse C+C++,
*Install Arduino Plugin,
And you should have your toolchain working.
I am happy this is finally on the road, no way back!
manu
PS: could a similar setup (using Eclipse) be made/explained for a STM or other mcu toolchains?

For STM I recommend CooCox ide + GCC-arm, simple and self contained.....plus you can get a Coos for rtos.

Hamburger · Post by **Hamburger** » Mon Jun 17, 2013 11:20 am

timecop wrote:Try using a newer avr-gcc than whats included with tarduino IDE - previous attempts at C-ifying the code have consistently produced smaller binaries. Also might want to try doing -Os or -O3 or similar for optimization options.

you are free to try that and whatever else you consider interesting.
But for the time being you must _not_ break compilation with arduino IDE.

alll · Post by **alll** » Mon Jun 17, 2013 12:08 pm

Hi,

Just to keep the same AVR-Arduino Eclipse-tool-chain initial approach and reuse big parts of the multiwii code, but now target a much more powerful 32-bit ARM Cortex-M3 microprocessor, could we not use these “Arduino-like” boards (The Maple Mini) and use external sensors for example?
http://leaflabs.com/

manu

timecop · Post by **timecop** » Mon Jun 17, 2013 12:30 pm

alll wrote:Hi,

Just to keep the same AVR-Arduino Eclipse-tool-chain initial approach and reuse big parts of the multiwii code, but now target a much more powerful 32-bit ARM Cortex-M3 microprocessor, could we not use these “Arduino-like” boards (The Maple Mini) and use external sensors for example?
http://leaflabs.com/

manu

Hello and welcome to 2 years ago.
Unfortunately, attempting to do development for a proper processor in a limited piece of shit that is tarduino IDE just doesn't work. so you either get with the program and use a proper toolchain/ide or stick with tarduino, which is what this thread is about.

Sebbi · Post by **Sebbi** » Mon Jun 17, 2013 1:48 pm

alll wrote:Hi,

Just to keep the same AVR-Arduino Eclipse-tool-chain initial approach and reuse big parts of the multiwii code, but now target a much more powerful 32-bit ARM Cortex-M3 microprocessor, could we not use these “Arduino-like” boards (The Maple Mini) and use external sensors for example?
http://leaflabs.com/

manu

That would be possible if MultiWii was actually using Arduino code. In reality MultiWii is heavily optimised code for some microcontrollers that happen to be used for Arduinos, too. If there were some kind of abstraction layer (tried some concepts here a few months back: https://github.com/sebastianherp/multiw ... experiment) it would be easier to port the code to different plattforms, I agree ;-)

clough42 · Post by **clough42** » Mon Jun 17, 2013 5:32 pm

casquad wrote:I have attached the MW project separated into cpp and h files.

This is exactly what I had imagined. Well done. I'm looking forward to having this as a branch in the repo.

casquad wrote:Btw, I have checked the svn and branches are not real branches, but direct commits into the "branches" folder

Branches are just as real in SVN as they are in any other SCM. SVN uses a unified version implementation where branches, tags and folders are all really the same thing internally. This leaves lots of flexibility for different branching and merging strategies, which is both a blessing and a curse. You have to pick a strategy for your project and operate it consistently, or you can make a pretty big mess pretty quickly--especially on a large project with tens of thousands of files and millions of lines of code.

In this case, though, a merge isn't really going to buy much, since almost all of the filenames have changed. When this goes into the _shared directory, SVN isn't going to be able to track changes across the boundary. With the exception of a couple of the .h files it'll just look like the .ino files were deleted and a bunch of new files were added.

casquad · Post by **casquad** » Mon Jun 17, 2013 8:55 pm

I agree that svn provides a great flexibility for code management. Nevertheless, there are great differences between real branching and commiting files to branches folder. First, when you commit new files, they are just that, new files. So if your trunk has a size of 1 MB, you checkout, modify and commit to some other branch, the size of the svn server is increased to 2 MB, while creating a new real branch does not increase the server load, because it is just a soft link. Only incremental changes are stored.

Second, the file changelog is lost, because files are commited as new files. With real branching you keep the complete log since each file was created. This is really usefull when you want to check what changes were introduced in that file before branching.

I work as a professional programmer for a multinational company. We are more than 200 developers in three countries and our last project has more than seven hundred thousand code lines. The branch-merge strategy is crucial for the success of the project, so it is the possibility of tracking the changes of a file, no matter in which branch that file is located (mainly to blame the programmer that introduced a bug) :-)

So even if svn has this great flexibility, I would suggest to follow some good practices in the use of the svn (for the good of the project).

I have some authentication problems in the svn to create the new branch, but as soon as it is solved, I will let you now so we can start working.

clough42 · Post by **clough42** » Mon Jun 17, 2013 9:27 pm

casquad wrote:So if your trunk has a size of 1 MB, you checkout, modify and commit to some other branch, the size of the svn server is increased to 2 MB, while creating a new real branch does not increase the server load, because it is just a soft link. Only incremental changes are stored.

Ahh...I think I see the confusion. Yes. If you check out, modify and then commit that copy into another path in the repository, it will indeed make a disconnected copy. This isn't usually how it's done, though. Usually, you would use the "svn copy" command to make a branch in the repository, which is just a new revision that links to the content and history of the original location. You would then check out (or switch to) the new branch and make incremental changes from there.

Unfortunately, SVN's use of the "svn copy" command for making a branch has created a lot of confusion.

And again, in this case, since the file set is changing, SVN won't be able to track the changes. Other, more sophisticated SCMs may handle this situation better, if they're aware of the file rename operations.

For those interested in learning more about SVN, the behavior is described pretty well here: http://svnbook.red-bean.com/en/1.7/svn.branchmerge.using.html

A quick excerpt from the above link:

Subversion's repository has a special design. When you copy a directory, you don't need to worry about the repository growing huge—Subversion doesn't actually duplicate any data. Instead, it creates a new directory entry that points to an existing tree. If you're an experienced Unix user, you'll recognize this as the same concept behind a hard link. As further changes are made to files and directories beneath the copied directory, Subversion continues to employ this hard link concept where it can. It duplicates data only when it is necessary to disambiguate different versions of objects..

alll · Post by **alll** » Mon Jun 17, 2013 9:52 pm

So, when will the ino files be abandoned?
I suppose you will "freeze" the current ino-development, "copy" the branch, reorganize to .h and .cpp, test all possible coptertest (1..5), if they pass, we continue with the new file structure. Am i right?
manu

casquad · Post by **casquad** » Mon Jun 17, 2013 10:02 pm

Yes, that is. I could not have explained it better :-)

The idea is create linked copies in the svn server side (what I called "real branches"), so we can easily track the changes and merge them. This also makes bug tracking really easy and also identifying which branches are affected.

I do not think we need to change the file set. I have never tried this, but after branching I think we could use the svn command "move" to rename the files while preserving the connection to the old ones and so the changelog. Could anybody confirm this?

Just a small comentary about this: what it is called here MultiWii and MultiWii_shared (both under trunk), from my point of view they are clearly two different branches, where MultiWii_shared could be the trunk and MultiWii could be the "stable" branch where we could easily merge from time to time the latest features implemented and already tested in the trunk.

And please do not missunderstand me. I do not want to despise the excelent work that is being done in this project. I just want to share my experience so we can keep improving and learning. I really think the project has a great potential :-)

clough42 · Post by **clough42** » Mon Jun 17, 2013 10:43 pm

casquad wrote:I do not think we need to change the file set. I have never tried this, but after branching I think we could use the svn command "move" to rename the files while preserving the connection to the old ones and so the changelog. Could anybody confirm this?

Yes. I can confirm that SVN will track history across file renames. I created a new file, made a change and committed, then renamed (with TortoiseSVN right-click Rename), committed and then committed another change under the new name. Here's the history that SVN is showing for the file:

It shows as a file delete and a file add, but it's tracking the changes across the revision. A file compare between r57898 and r57900 shows the textual changes correctly.

This is SVN 1.7. I don't have commit rights to test in the MultiWii repo.

Sebbi · Post by **Sebbi** » Tue Jun 18, 2013 10:58 am

Is there a timeframe for this project? I'd very much like to persue my quest to have some form of abstraction layer in between, so the MultiWii code can run on different hardware (like some ARM boards which a both cheaper and faster than plain old Arduinos) ;-)

casquad · Post by **casquad** » Tue Jun 18, 2013 11:57 am

Could anoyone help me to create a new branch in the svn? I have been trying hundred times and I am giving up. I tried Tortoise gui and command line, both failed. This is what I do:

svn copy https://multiwii.googlecode.com/svn/trunk https://multiwii.googlecode.com/svn/bra ... pMigration -m "New branch for cpp/h migration" --username codetweaker83@gmail.com

This command has always worked for me in other projects. After pressing enter, it asks for password (I use the one generated from GoogleCode, not my account password), but it fails with no error, just promp for another username and password again and again. I think I have permission to create branches, but I am starting to doub it. I have check the repo history and there is one real branch "DaedalusFC", but I do now know how it was created...

clough42 · Post by **clough42** » Tue Jun 18, 2013 5:22 pm

casquad wrote:Could anoyone help me to create a new branch in the svn? I have been trying hundred times and I am giving up. I tried Tortoise gui and command line, both failed. This is what I do:

svn copy https://multiwii.googlecode.com/svn/trunk https://multiwii.googlecode.com/svn/bra ... pMigration -m "New branch for cpp/h migration" --username codetweaker83@gmail.com

This command has always worked for me in other projects. After pressing enter, it asks for password (I use the one generated from GoogleCode, not my account password), but it fails with no error, just promp for another username and password again and again. I think I have permission to create branches, but I am starting to doub it. I have check the repo history and there is one real branch "DaedalusFC", but I do now know how it was created...

Your command line looks right to me. I just tested on my own googlecode project, and was able to create a branch with:

svn copy https://boisefirstmediasuite.googlecode.com/svn/trunk https://boisefirstmediasuite.googlecode ... pMigration -m "New branch for cpp/h migration" --username clough42@gmail.com

You appear to have the same permissions in the MultiWii project as Guillaume, who created the DaedalusFC branch.

casquad · Post by **casquad** » Tue Jun 18, 2013 9:08 pm

Ok, I finally managed to create the branch. You can find it under:

https://multiwii.googlecode.com/svn/bra ... Migration/

The MultiWii_shared project has already been ported to cpp/h files. It is compiling in Arduino IDE and you can also create a project in Eclipse to compile it. I have only checked compilation with QUADX and MultiWiiMega board. Code size is a bit larger because some inline optimizations cannot be applied when compiling separately. Please feel free to checkout, test it and add your contributions. As you can see, the svn history of each file is preserved event though some of them have been renamed.

As first work to do, I would say testing the compilation with all board configurations and check the GPS and LCD files.