l i n u x - u s e r s - g r o u p - o f - d a v i s
L U G O D
 
Next Meeting:
August 5: Social gathering
Next Installfest:
TBD
Latest News:
Jul. 4: July, August and September: Security, Photography and Programming for Kids
Page last updated:
2008 Jan 15 14:25

The following is an archive of a post made to our 'vox-tech mailing list' by one of its subscribers.

Report this post as spam:

(Enter your email address)
Re: [vox-tech] collaborative data storage (of excel files)
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [vox-tech] collaborative data storage (of excel files)



On Jan 15, 2008, at 10:33 AM, Dylan Beaudette wrote:

On Tuesday 15 January 2008, Henry House wrote:
On 2008-01-15, wrote Dylan Beaudette:
Hi,

some of the people in my lab are interested in collaboratively compiling
a large quantity of environmental data- each user appending several
hundred measurements of several variables every week.

They are currently emailing around a spread sheet file and there have
been numerous data accidents. Now they are asking to put the file onto a
shared drive, so that they can access it remotely. This sounds like a
terrible idea to me- even worse than the previous attempt.
Their idea is distilled evil! But you already knew that.
Indeed!


The data are essentially rows and cols of numbers that are added to and
edited weekly.

At first I thought subversion might be helpful, but revision control
doesn't work so well with binary data (excel files)... unless there is
something I don't know about. It would be hard to detect conflicts, or to
merge data. However, it would allow for timestamps and revision numbers
to provide some level of authority.

Designing some kind of database-driven system seems like a logical
choice, but I do not have the time to do this. Perhaps there is already
something out there.

Does anyone have some insight into how to solve this data management
nighmare?
The right way to do this is to use a database. But, an easier
maybe-almost-as-good solution might be to use subversion and save the
data as CSV text files (excel can do this just fine). It is useful to
add comment lines (maybe you could have an internal convention about
this) that help subversion to figure out where to merge in changes.
This is a good idea. Unfortunately we might be stuck with 4 or 5 mega CSV
files which are constantly appended to, but SVN should be able to deal with
keeping things sane. The trick will be to get people to realize that these
are CSV files, and therefore no monkeying around with formulas, etc.

OpenOffice.org provides a database tool that would probably work for you. You can then use the database data in spreadsheet formulas. Since you have only one table (for now) it should be easy to design and create the database. FileMaker is better, but costs money.

Either way, you'll get admin-free database work on a shared file, but they're not geared towards merging different copies that people are working on off-line.

I advocate separating the data and code - some recent studies point out that bugs in Excel code are actually a widespread and expensive problem. As you pointed out, even just having a version-controlled CSV file and linking it in to a spread sheet should provide some protection.

The next step up, sorta, might be to set up a web server with a very simple CGI for appending records to the CSV and committing, accessed via a web form. From there you could proceed to using an low-admin database if you wanted.

-Harold

I'll keep digging around for ideas.

Dylan

--
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
_______________________________________________
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech
_______________________________________________
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech



LinkedIn
LUGOD Group on LinkedIn
Sign up for LUGOD event announcements
Your email address:
facebook
LUGOD Group on Facebook
'Like' LUGOD on Facebook:

Hosting provided by:
Sunset Systems
Sunset Systems offers preconfigured Linux systems, remote system administration and custom software development.

LUGOD: Linux Users' Group of Davis
PO Box 2082, Davis, CA 95617
Contact Us

LUGOD is a 501(c)7 non-profit organization
based in Davis, California
and serving the Sacramento area.
"Linux" is a trademark of Linus Torvalds.

Sponsored in part by:
EDGE Tech Corp.
For donating some give-aways for our meetings.