Working with big .txt files
This week we had to work with a fairly large text file weighing in at over 220mb and containing over 1.9million records from NZ Post. I found that even the 64Bit version of Microsoft Excel wasn’t very happy dealing with that many records in one go. The best and quickest solution for us was to split up the original text file in a number of smaller files, import and edit them in separate sheets and then combine all the edited records in a single sheet ready to save to .csv format. There are number of great programs on the market that can do this for you but we found that Gsplit worked really well for this particular job. It had one great thing going for it from the outset which was the price, Free. I think the program is usually used for splitting large files up into smaller pieces for nefarious purposes (filesharing films etc) but in this case it works really well splitting up the text file and automatically putting a new header into each file. It took seconds to split up the file smaller 45mb files which were easy to import and work with in Excel.
The GUI was (fairly) simple to use. I did manage to split it up into the wrong format once or twice but that was mainly because I didn’t take notice of the very obvious instructions, so more user error and than product error.
All in all would definitely recommend.