Las imagenes van llegando poco a poco y se han de ir procesando paulatinamente. Lo primero es saber cual de las imagenes que han subido no han sido procesadas,

$ ls /home/data/subjects/facehbi_smc0*/stats/aseg.stats | awk -F"_smc" {'print $2'} | awk -F"/" {'print $1'} > mri_hechos.txt
$ sed 's/0/F/' mri_hechos.txt > mri_hechos.dir
$ ls /nas/corachan/facehbi/ > mri_up.dir
$ grep -v "`cat mri_hechos.dir`" mri_up.dir > yet.dir
$ for a in `cat yet.dir`; do for x in -d /nas/corachan/facehbi/$a/*; do if [[ `dckey -k "SeriesDescription" $x/Img00001.dcm 2>&1 | grep t1_mprage` ]]; then dcm2nii -o tmp/ $x/Img00001.dcm; fi; done; mkdir processed/$a; mv tmp/* processed/$a; done
$ sed 's/F/0/;s/\(.*\)/\1;smc/' mri_hechos.dir > /nas/facehbi/wave1_mri.csv
$ sed 's/F/0/;s/\(.*\)/\1;smc/' yet.dir > /nas/facehbi/wave2_mri.csv
$ cat /nas/facehbi/wave1_mri.csv /nas/facehbi/wave2_mri.csv > /nas/facehbi/facehbi.csv
$ for a in `ls  processed/F*/*.nii.gz`; do b=$(echo $a | sed s'/processed\/F/\/nas\/facehbi\/mri\/smc0/; s/\/s0/s00/; s/a1001//'); cp $a $b; done
$ pfsl2fs.pl -cut wave2_mri.csv facehbi

DELETEME

$ for a in `cat yet.txt`; do cp -r /nas/corachan/facehbi/$a /nas/raw_images/facehbi/mri/; done
$ for a in `cat yet.txt`; do for x in -d /nas/raw_images/facehbi/mri/$a/*; do if [[ `dckey -k "SeriesDescription" $x/Img00001.dcm 2>&1 | grep t1_mprage` ]]; then dcm2nii -o tmp/ $x/Img00001.dcm; fi; done; mkdir processed/$a; mv tmp/* processed/$a; done

$ ls -d data/subjects/facehbi* | awk -F"smc" {'print $2'} > ~/data/facehbi/fs_done.txt
$ ls -d /nas/corachan/facehbi/* | awk -F"/" {'print $5'} | sed 's/F/0/' > ~/data/facehbi/haymri.txt
$ grep -v "`cat ~/data/facehbi/fs_done.txt`" ~/data/facehbi/haymri.txt

$ cp processed/F042/s005a1001.nii.gz /nas/facehbi/mri/smc0042s0005.nii.gz

$ for a in `ls  processed/F*/*.nii.gz`; do b=$(echo $a | sed s'/processed\/F/\/nas\/facehbi\/mri\/smc0/; s/\/s0/s00/; s/a1001//'); echo "$a $b"; done

$ sed 's/F/0/;s/\(.*\)/\1;smc/' yet.txt > /nas/facehbi/wave2_mri.csv

$ pfsl2fs.pl -cut wave2_mri.csv facehbi

$ mv facehbi.csv wave1_mri.csv
$ cat wave1_mri.csv wave2_mri.csv > facehbi.csv

Ahora ya se podria procesar con Freesurfer

$ precon.pl -cut wave2_mri.csv facehbi

$ for a in `ls /nas/corachan/facehbi/`; do for x in -d /nas/corachan/facehbi/$a/*; do if [[ `dckey -k "SeriesDescription" $x/Img00001.dcm 2>&1 | grep "ep2d_diff_mddw_64_p2$"` ]]; then dcm2nii -o /nas/facehbi/tmp_dti/ $x/; fi; done; mkdir processed/$a; mv tmp_dti/* processed/$a; done
$ cd /nas/facehbi
$ for x in `find processed/ -name "*.bval"`; do nm=$(echo $x | sed 's/processed\/F\(.*\)\/s\(.*\)a001.bval/smc0\1s0\2/'); cp $x dti/${nm}.bval;  cp ${x%.bval}.bvec dti/${nm}.bvec; cp ${x%.bval}.nii.gz dti/${nm}.nii.gz; done 

$ cat acqparams.txt 
0 1 0 0.12192
0 -1 0 0.12192
 
$ indx=""
$ for ((i=1; i<=143; i+=1)); do indx="$indx 1"; done
$ echo $indx > dti_index.txt
 
$ dti_reg.pl -nocorr facehbi
 
$ dti_metrics.pl facehbi
$ dti_metrics.pl -a1 facehbi
$ dti_metrics_custom.pl -uofm SN facehbi
$ dti_metrics_custom.pl -uofm SN_anterior facehbi
 
$ dti_track.pl -uofm DMN facehbi
$ for x in `ls -d working/smc0*_probtrack_out`; do mv $x `echo $x | sed 's/out/DMN/'`;done
$ dti_metrics_alt.pl -path DMN facehbi

Para ir procesando según vayan llegando las imágenes,

$ cp -ru clinic/* raw_images/facehbi/fbb/
$ ls raw_images/facehbi/fbb/ | sed 's/FACEHBI-F\(.*\)B/\1/' > facehbi/fbb_copied.txt
$ grep -v "`cat facehbi/fbb_done.txt`" facehbi/fbb_copied.txt > facehbi/fbb_wave2.txt
$ grep -v "`cat facehbi/fbb_done.txt`" facehbi/fbb_copied.txt | sed 's/\(.*\)/0\1;smc/' > facehbi/wave2_fbb.csv
$ sed 's/\(.*\)/0\1;smc/' facehbi/fbb_copied.txt > facehbi/wave1_fbb.csv
$ sed 's/\(.*\)/0\1;smc/' facehbi/fbb_copied.txt > facehbi/all_fbb.csv

Otro ejemplo puede ser, si ya tengo hecho un pedazo, ¿Como saco de las que no he procesado? Pues así

$ ls /nas/clinic/ | sed 's/FACEHBI.*F\(.*\)B.*/0\1/' | sort | uniq > hayfbb.txt
$ awk -F";" {'print $1'} facehbi_fbb_fs_suvr.csv > hechos.txt
$ grep -v "`cat hechos.txt`" hayfbb.txt | sed  's/\(.*\)/\1;smc/' > yet.txt

Los DICOM llegan en carpetas de hasta 500 imagenes DICOM. Esto quiere decir que hay que copiar previamente todas las imagenes que queremos convertir a una misma carpeta y a partir de ahi es que se convierten.

Si tenemos un archivo txt (yet.txt en el ejemplo) y sabemos cual imagen hay que buscar hacemos,

$ for x in `awk -F";" {'print $1'} yet.txt`; do y=/nas/clinic/FACEHBI-F$(echo $x|sed 's/0\(.*\)/\1/')B/DICOM/; for od in `find $y -maxdepth 2 -name "*0000" -o -name "*0001"`; do for okf in `ls $od`; do if [[ `dckey -k "SeriesDescription" $od/$okf 2>&1 | grep  "5min"` ]] ; then cp $od/$okf /nas/facehbi/tmp_2nifti/; fi; done;ff=$(ls /nas/facehbi/tmp_2nifti/ | head -n 1); dcm2nii -o /nas/facehbi/tmp/ /nas/facehbi/tmp_2nifti/$ff; done; rm -rf /nas/facehbi/tmp_2nifti/*; nf=$(ls /nas/facehbi/tmp/*.nii.gz | head -n 1); fslsplit $nf /nas/facehbi/fbb_ok/smc${x}s; rm -rf /nas/facehbi/tmp/*; done

Esto está mejor aqui

lo que sale del omi es una mierda y hay que convertirlo en algo potable:

$ awk -F"," '{printf "%04d, %s, %d, %s, %s\n", $7,$9,$4,$3,$8}' mierda_demo.csv | sed 's/Varón/1/;s/Mujer/0/;s/, \([0-9]*\)\/\([0-9]*\)\/\([0-9]*\)/, \3-\2-\1/g' | awk -F"," '{printf "%04d, %d, %d, %d\n", $1,$2,$3,$5-$4}' > mier_demo.csv

y sacamos la edad de la fecha de nacimiento y la del screening

#!/usr/bin/env perl 
 
use strict;
use warnings;
use utf8;
use Date::Manip;
use Math::Round;
 
my $ifile = shift;
my $heads;
open IDF, "<$ifile" or die "Couldn't find input file";
while(<IDF>){
	if(/^Subject.*/){
		print "Subject, Gender, Education, Age\n";
	}
	if(my ($nodate, $date1, $date2) = /(\d{4}, \d+, \d{1,2}), (\d+-\d+-\d+), (\d+-\d+-\d+)/){
		my $age  = nearest(0.1, Delta_Format(DateCalc($date1,$date2),"%hv")/(24*365.2425));
		print "$nodate, $age\n";
	}
}
close IDF;

First approach,

[osotolongo@detritus facehbi]$ for y in /nas/corachan/facehbi/*; do for x in ${y}/*; do if [[ `dckey -k "SeriesDescription" ${x}/Img00001.dcm 2>&1 | grep t1_mprage` ]]; then a=$(dckey -k "StudyDate" ${x}/Img00001.dcm 2>&1); fi; done; echo ${y} ${a};done
/nas/corachan/facehbi/F001 20141205
/nas/corachan/facehbi/F002 20141205
/nas/corachan/facehbi/F003 20141211
/nas/corachan/facehbi/F004 20141212
.......

Vamos a ordenarlo un poco,

[osotolongo@detritus facehbi]$ for y in /nas/corachan/facehbi/*; do for x in ${y}/*; do if [[ `dckey -k "SeriesDescription" ${x}/Img00001.dcm 2>&1 | grep t1_mprage` ]]; then a=$(dckey -k "AcquisitionDate" ${x}/Img00001.dcm 2>&1); fi; done; echo ${y} ${a};done | sed 's/.*F/0/; s/ /;/' > /home/osotolongo/facehbi/dicom_mri_v0_date.csv

O mas rapido,

[osotolongo@detritus facehbi]$ for y in /nas/corachan/facehbi_2/*; do for x in `find ${y} -type f | head -n 1`; do a=$(dckey -k "AcquisitionDate" ${x} 2>&1); done; echo ${y} ${a}; done | sed 's/.*F/0/; s/_.* / /; s/ /;/' | uniq | grep -v 0ACEHBIS > /home/osotolongo/facehbi/dicom_mri_v2_date.csv

one random file by subject,

[osotolongo@detritus facehbi]$ find /nas/clinic/facehbi/FACEHBI-F001B/DICOM/ -type f | head -n 1
/nas/clinic/facehbi/FACEHBI-F001B/DICOM/15012118/23390000/27112349

ahora,

[osotolongo@detritus facehbi]$ for y in /nas/clinic/facehbi/*; do for x in `find ${y}/DICOM/ -type f | head -n 1`; do a=$(dckey -k "AcquisitionDate" ${x} 2>&1); done; echo ${y} ${a}; done | sed 's/.*-F/0/; s/B//; s/ /;/' > /home/osotolongo/facehbi/dicom_fbb_v0_date.csv
[osotolongo@detritus facehbi]$ for y in /nas/clinic/facehbi_2/*; do for x in `find ${y}/ -type f | head -n 1`; do a=$(dckey -k "AcquisitionDate" ${x} 2>&1); done; echo ${y} ${a}; done | grep -v Error | sed 's/_/-/g; s/.*-F/0/; s/F//; s/ /;/' > /home/osotolongo/facehbi/dicom_fbb_v2_date.csv

Parece raro pero dado el poco consistente formato de los archivos hay que cambiar las ordenes para cad directorio.

Las fechas quedan ahora en cuatro archivos,

[osotolongo@detritus facehbi]$ ls -l dicom_*
-rw-rw---- 1 osotolongo osotolongo 2800 May 20 10:06 dicom_fbb_v0_date.csv
-rw-rw---- 1 osotolongo osotolongo 2786 May 20 10:14 dicom_fbb_v2_date.csv
-rw-rw---- 1 osotolongo osotolongo 2884 May 18 14:59 dicom_mri_v0_date.csv
-rw-rw---- 1 osotolongo osotolongo 3262 May 20 10:02 dicom_mri_v2_date.csv

Voy a hacer un parser para juntar todo,

date_parser.pl

#!/usr/bin/perl
#
#use strict;
#use warnings;
 
my %fdates = ( FBBv0 => "dicom_fbb_v0_date.csv",
        FBBv2 => "dicom_fbb_v2_date.csv",
        MRIv0 => "dicom_mri_v0_date.csv",
        MRIv2 => "dicom_mri_v2_date.csv",
);
 
my $fdpath = '/home/osotolongo/facehbi/';
my %imgdates;
 
foreach my $fdate (sort keys %fdates){
        $real_file = $fdpath.$fdates{$fdate};
        open IDF, "<$real_file" or die "No such file";
        while(<IDF>){
                (my $subject, my $imdate) = /(.*);(.*)/;
                $imgdates{$subject}{$fdate} = $imdate;
        }
        close IDF;
}
print "Subject";
foreach my $fdate (sort keys %fdates){
        print ", $fdate";
}
print "\n";
foreach my $subject (sort keys %imgdates){
        print "$subject";
        foreach my $fdate (sort keys %fdates){
                if (exists($imgdates{$subject}{$fdate})){
                        print ", $imgdates{$subject}{$fdate}";
                }else{
                        print ", -";
                }
        }
        print "\n";
}

y ahi va,

[osotolongo@detritus facehbi]$ ./date_parser.pl > dicom_dates.csv
[osotolongo@detritus facehbi]$ head dicom_dates.csv
Subject, FBBv0, FBBv2, MRIv0, MRIv2
0001, 20141211, 20170126, 20141205, 20170124
0002, 20141211, 20170420, 20141205, 20170323
0003, 20141218, 20170126, 20141211, 20170123
0004, 20141218, -, 20141212, -
0005, 20150122, 20170202, 20150107, 20170123
0006, 20150115, 20170126, 20141223, 20170124
0007, 20150115, 20170126, 20141219, 20170120
0008, 20150115, 20170202, 20141220, 20170125
0009, 20150129, 20170216, 20150110, 20170207