Import 10GB txt file (is there restrictions)

Sergej

Hi!
Is it possible to import data from 10GB txt file containing 200 000 rows in awareim?
I tried but it didnt imported at all. say Data not imported.

tford

You should probably try IN BATCHES OF ...

Sergej

I mistyped there are 200 million rows.
I am trying to import using Standard menu->Import
then there is In batches of 1000.

I cannot open this file in any program, I thought that awareim could open it because it works with databases.

tford

I have no experience with data of that magnitude.

IMPORT action can be used in a process where you can better control the IN BATCHES OF number.

pbrad

Are you trying to import this file because you want the records in your application or simply in order to view/access the contents of the file?
Pete

Sergej

In order view/access the contents of the file and to filter the data.
The only program in what I could open it is Emeditor but there is no filtering.

Tom's advice helped, dedicated process works whet you specify direct location to the file but not using upload.

But it was working for about 1-2 hours and failed - connection error, session timed out.
It imported 700 000 rows. i am very surprised and glad that awareim could do that, I will try it again.
But I now calculated that it imported less than 1% of all data so import can last for 100 hours !!! 🙂

pbrad

You might want to check out vedit.com

Cheers,
Pete

aware_support

Import from UI will not do it because it has one hour timeout limit. The process should do it, though, but it will take a while (to put it mildly)

Sergej

Could you advice please how to make it not from UI?

aware_support

Use a process that uses the IMPORT action. Make the process run on a background by choosing the corresponding process option.