Welcome to KORG!

There are two public functions - QSort that will sort the data, and Uniq() that will remove all dups. Both of these functions work with data in an array. The FileIO() function can load your file into an array.

I'm not sure about the file size as that is quite hefty. Here's the basic process:
 Code:
$aFileData = FileIO('mybigfile.csv', 'R')   ; read the big CSV file
$aSort = QSort($aFileData)  ; sort the array
$aUniq = Uniq($aSort) ; remove dups
$aFileData = FileIO('mynewbigfile.csv', 'W', $aUniq) ; write a new, sorted, deduped file
Glenn
_________________________
Actually I am a Rocket Scientist! \:D