Randomize lines in two files, keeping relative order

I recently wanted to randomize the lines in two files, but to keep the relative order of the lines between the files. So I can remember how to do this next time I will post it here.

for i in `cat file1.txt`;do echo $RANDOM;done >randomOrder.txt
paste randomOrder.txt  file1.txt file2.txt  |sort -k1n >sorted.txt
cut -f 2 sorted.txt  >file1.txt
cut -f 3 sorted.txt  >file2.txt
rm -f sorted.txt
rm -f randomOrder.txt

BASH: randomize the lines in a file

I colleague needed to randomize the lines in a text file, and as usual google as the answer. I removed the sed and replaced it with cut. It works due to the $RANDOM variable which returns a psedu-random number each time you call it. Nice.

for i in `cat textFile.csv`;do echo "$RANDOM $i";done |sort -n -k 1|cut -f 2- -d " "

So it adds a random number before each line, then sorts on this number. Simple but clever.

Perl one liner: Random Lines from a File

I have some bed files that are too large to process in a reasonable time, so I need to randomly sample lines from them to create files of a workable size.

I used some bash and perl magic for this.

for f in *.bed;do export WC=`wc ${f} -l |cut -f 1 -d " "`;perl -i -ne 'srand;print if rand() <1500/$ENV{'WC'}' ${f} ;done

Basically, it checks the length of the file and stores the result in the environment variable WC, then it reads in the file line by line and only prints out the line if a random number between 0 and 1 is less than the proportion of our required size (1500 in this case) of our length (WC).

This is looped round all bed files in the current directory.

Edit:
You could also do something like this:

perl -ne 'print rand;print "\t";print;' FILENAME |sort |head -n 100 |cut -f 2 >NEWFILENAME

Which will return a random 100 lines from the file.