Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » Software » comp.soft-sys.matlab

Topic: really big data files
Replies: 4   Last Post: Nov 17, 2009 10:21 AM

Advanced Search
Reply to this Topic Reply to this Topic

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Jon Shultz

Posts: 5
Registered: 8/28/09
really big data files
Posted: Nov 8, 2009 2:24 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

I'm trying to read in a datafile that's really big (>2GB) in sections that are a couple hundred thousand lines long each. I need to know how many lines are in the parent file first.

I have a routine now that does it like this:
totlines=0;
while ~feof(fid)
line=fgetl(fid);
totlines=totlines+1;
end

This does well with the memory part, but takes forever. There has got to be a more efficient way to do this, but I'm stuck.

Thanks!

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2009. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Goodwin College of Professional Studies.