Differences

This shows you the differences between two versions of the page.

--- neuroimagen:cpac [2019/05/16 09:31] – osotolongo
+++ neuroimagen:cpac [2020/08/04 10:58] (current) – external edit 127.0.0.1
@@ Line 3: / Line 3: @@
 Una opcion para preprocesar las imagenes fMRI es utilizar [[http://fcp-indi.github.io/docs/user/index.html|C-PAC]].
-Se instala bajando una imagen [[https://www.docker.com/|docker]] que he transformado a [[https://www.sylabs.io/|Singularity]]. Siguiendo [[http://fcp-indi.github.io/docs/user/quick.html#running-on-singularity|la documentacion]] puede lanzarse como,
+Se instala bajando una imagen [[https://www.docker.com/|docker]] que he transformado a [[https://www.sylabs.io/|Singularity]].
+===== Sujetos individuales =====
+Siguiendo [[http://fcp-indi.github.io/docs/user/quick.html#running-on-singularity|la documentacion]] puede lanzarse como,
 <code bash>
@@ Line 30: / Line 33: @@
 </code>
-La orden ahora es ligeramente diferente,
+La orden ahora es ligeramente diferente pues he de especificar el archivo de configuracion del pipeline,
 <code>
@@ Line 37: / Line 40: @@
 </code>
+minutos después,
+<code>
+    End of subject workflow resting_preproc_sub-0002_ses-1
+    CPAC run complete:
+        Pipeline configuration: analysis
+        Subject workflow: resting_preproc_sub-0002_ses-1
+        Elapsed run time (minutes): 184.085074282
+        Timing information saved in /outputs/log/cpac_individual_timing_analysis.csv
+        System time of start:      2019-05-16 09:17:52
+        System time of completion: 2019-05-16 12:21:49
+</code>
+El output es enorme,
+<code>
+[osotolongo@brick01 mopead]$ ls cpac_out/output/pipeline_analysis_nuisance/sub-0002_ses-1/
+alff_to_standard_smooth_zstd                    frame_wise_displacement_power
+anatomical_brain                                functional_brain_mask
+anatomical_csf_mask                             functional_brain_mask_to_standard
+anatomical_gm_mask                              functional_freq_filtered
+anatomical_reorient                             functional_nuisance_regressors
+anatomical_to_mni_nonlinear_xfm                 functional_to_anat_linear_xfm
+anatomical_to_standard                          functional_to_standard
+anatomical_to_symmetric_mni_nonlinear_xfm       mean_functional_to_standard
+anatomical_wm_mask                              mni_to_anatomical_nonlinear_xfm
+ants_affine_xfm                                 motion_correct
+ants_initial_xfm                                motion_params
+ants_rigid_xfm                                  qc
+ants_symmetric_affine_xfm                       qc_html
+ants_symmetric_initial_xfm                      roi_timeseries
+ants_symmetric_rigid_xfm                        spatial_map_timeseries
+centrality_smooth_zstd                          spatial_map_timeseries_for_DR
+dr_tempreg_maps_files_to_standard_smooth        symmetric_anatomical_to_standard
+dr_tempreg_maps_zstat_files_to_standard_smooth  symmetric_mni_to_anatomical_nonlinear_xfm
+falff_to_standard_smooth_zstd                   vmhc_fisher_zstd_zstat_map
+frame_wise_displacement_jenkinson
+</code>
+Afortunadamente, [[http://fcp-indi.github.io/docs/user/output_dir.html|la explicacion de cada directorio esta documentada]].
+==== Integrando en el cluster ====
+El esquema para correr en el cluster es el de lanzar las imagenes de singularity en paralelo. Hay que hacer varias  pruebas por si se solapan unas a otras pero en principio intentaremos lanzar unos 8 procesos por nodo (o menos).
+++++ Aqui el codigo |
+<code perl cpac.pl>
+#!/usr/bin/perl
+# Copyright 2019 O. Sotolongo <asqwerty@gmail.com>
+use strict; use warnings;
+use File::Find::Rule;
+use NEURO qw(print_help get_pair load_study achtung shit_done get_lut check_or_make centiloid_fbb);
+use Data::Dump qw(dump);
+use File::Remove 'remove';
+use File::Basename qw(basename);
+my $cpac_img = '/nas/software/cpac-latest.simg';
+my $pipe_conf = 'cpac_pipeline_config.yml';
+my $lib_conf = $ENV{'PIPEDIR'}.'/lib/'.$pipe_conf;
+my $cfile;
+@ARGV = ("-h") unless @ARGV;
+while (@ARGV and $ARGV[0] =~ /^-/) {
+    $_ = shift;
+    last if /^--$/;
+    if (/^-cut/) { $cfile = shift; chomp($cfile);}
+    if (/^-h$/) { print_help $ENV{'PIPEDIR'}.'/doc/cpac.hlp'; exit;}
+}
+my $study = shift;
+unless ($study) { print_help $ENV{'PIPEDIR'}.'/doc/cpac.hlp'; exit;}
+my %std = load_study($study);
+my $w_dir = $std{'WORKING'};
+my $data_dir = $std{'DATA'};
+my $bids_dir = $data_dir.'/bids';
+my $fmriout_dir = $data_dir.'/cpac_out';
+check_or_make($fmriout_dir);
+my $outdir = "$std{'DATA'}/slurm";
+check_or_make($outdir);
+my $tmpdir = "$std{'DATA'}/ctmp";
+check_or_make($tmpdir);
+my $proj_conf = $data_dir.'/'.$pipe_conf;
+system("cp $lib_conf $proj_conf") unless (-e $proj_conf);
+my @subjects;
+if($cfile){
+	open DBF, $cfile or die "No such file\n";
+	while(<DBF>) {
+		chomp;
+		push @subjects, $_;
+	}
+	close DBF;
+}else{
+	opendir DBD, $bids_dir or die "Cold not open dir\n";
+	while (my $thing = readdir DBD){
+		if ($thing eq '.' or $thing eq '..') {
+			next;
+		}
+		if ($thing =~ /sub-*/) {
+			push @subjects, $thing;
+		}
+	}
+	closedir DBD;
+}
+foreach my $subject (@subjects) {
+	my $orderfile = $outdir.'/'.$subject.'_cpac.sh';
+	open ORD, ">$orderfile";
+	print ORD '#!/bin/bash'."\n";
+	print ORD '#SBATCH -J cpac_'.$study."\n";
+	print ORD '#SBATCH --time=72:0:0'."\n"; #si no ha terminado en X horas matalo
+	print ORD '#SBATCH --mail-type=FAIL,TIME_LIMIT,STAGE_OUT'."\n"; #no quieres que te mande email de todo
+	print ORD '#SBATCH -o '.$outdir.'/cpac-%j'."\n";
+	print ORD '#SBATCH -c 20'."\n";
+	print ORD '#SBATCH -p fast'."\n";
+	print ORD '#SBATCH --mail-user='."$ENV{'USER'}\n";
+	print ORD 'srun singularity run --cleanenv -B '.$data_dir.':/project -B '.$bids_dir.':/bids_dataset -B '.$fmriout_dir.':/outputs -B '.$tmpdir.':/scratch '.$cpac_img.' --pipeline_file /project/'.$pipe_conf.' /bids_dataset /outputs participant --participant_label '.$subject."\n";
+	close ORD;
+	system("sbatch $orderfile");
+	sleep(20);
+}
+my $orderfile = $outdir.'/cpac_end.sh';
+open ORD, ">$orderfile";
+print ORD '#!/bin/bash'."\n";
+print ORD '#SBATCH -J cpac_'.$study."\n";
+print ORD '#SBATCH --mail-type=END'."\n"; #email cuando termine
+print ORD '#SBATCH --mail-user='."$ENV{'USER'}\n";
+print ORD '#SBATCH -p fast'."\n";
+print ORD '#SBATCH -o '.$outdir.'/cpac_end-%j'."\n";
+print ORD ":\n";
+close ORD;
+my $order = 'sbatch --dependency=singleton '.$orderfile;
+exec($order);
+</code>
+++++
+**Nota**: Al correr //c-pac// en el cluster dan errores de I/O. Esto puede ser debido a un intenso uso de la red o de disco. He de reducir el nuemro de lanzamientos concurrentes y probar de nuevo.
+=== Output ===
+//C-PAC// genera un directorio de output por cada sujeto analizado
+<code>
+[osotolongo@detritus mopead]$ ls cpac_out/output/pipeline_analysis_nuisance/
+sub-0001_ses-1  sub-0003_ses-1  sub-0005_ses-1  sub-0007_ses-1  sub-0009_ses-1  sub-0012_ses-1  sub-0015_ses-1  sub-0018_ses-1  sub-0020_ses-1  sub-0022_ses-1
+sub-0002_ses-1  sub-0004_ses-1  sub-0006_ses-1  sub-0008_ses-1  sub-0010_ses-1  sub-0014_ses-1  sub-0017_ses-1  sub-0019_ses-1  sub-0021_ses-1  sub-0023_ses-1
+</code>
+[[http://detritus.fundacioace.com/files/example_cpac_out.txt|El output completo para cada sujeto se puede ver aqui]].
+{{:neuroimagen:2019-05-30-112300_grim.png|}}
+=== Errores ===
+Los errores que se obtienen son muy diversos y parece indicar que tienen que ver mas con la calidad de la adquisicion que con el software en si mismo.No obstante al fallar en distintos puntos del procesamiento es posible que algunos de los sujetos fallidos contengan los archivos de output para algun postprocesdo especifico. En cada caso se ha de buscar si existen estos archivos.
+<code>
+-07:56:45,394 nipype.workflow INFO:
+         [Node] Setting-up "resting_preproc_sub-0004_ses-1.sinker_0_13" in "/scratch/working/resting_preproc_sub-0004_ses-1/sinker_0_13".
+-07:56:46,61 nipype.workflow INFO:
+         [Node] Running "sinker_0_13" ("nipype.interfaces.io.DataSink")
+-07:56:46,617 nipype.workflow INFO:
+         [Node] Finished "resting_preproc_sub-0004_ses-1.sinker_0_13".
+srun: error: brick01: task 0: Exited with exit code 1
+</code>
+<code>
+-08:02:22,31 nipype.workflow INFO:
+         [Node] Outdated cache found for "resting_preproc_sub-0003_ses-1.qc_snr_0.std_dev_anat".
+-08:02:22,347 nipype.workflow INFO:
+         [Node] Running "std_dev_anat" ("nipype.interfaces.fsl.preprocess.ApplyWarp"), a CommandLine Interface with command:
+applywarp --in=/scratch/working/resting_preproc_sub-0003_ses-1/qc_snr_0/_scan_rest_run-1/std_dev/sub-0003_task-rest_run-1_bold_calc_tshift_resample_volreg_calc_maths_tstat.nii.gz --ref=/scratch/working/resting_preproc_sub-0003_ses-1/anat_preproc_bet_0/anat_skullstrip_orig_vol/sub-0003_T1w_resample_calc.nii.gz --out=/scratch/working/resting_preproc_sub-0003_ses-1/qc_snr_0/_scan_rest_run-1/std_dev_anat/sub-0003_task-rest_run-1_bold_calc_tshift_resample_volreg_calc_maths_tstat_warp.nii.gz --premat=/scratch/working/resting_preproc_sub-0003_ses-1/func_to_anat_bbreg_0/_scan_rest_run-1/bbreg_func_to_anat/sub-0003_task-rest_run-1_bold_calc_tshift_resample_volreg_calc_tstat_flirt.mat --interp=trilinear
+-08:02:25,845 nipype.workflow INFO:
+         [Node] Finished "resting_preproc_sub-0003_ses-1.qc_snr_0.std_dev_anat".
+srun: error: brick01: task 0: Exited with exit code 1
+</code>
+<code>
+nipype.external.portalocker.LockException: (5, 'Input/output error')
+-08:02:21,718 nipype.workflow INFO:
+         [Node] Setting-up "resting_preproc_sub-0009_ses-1.qc_skullstrip_1.montage_skull.montage_a" in "/scratch/working/resting_preproc_sub-0009_ses-1/qc_skullstrip_1/montage_skull/montage_a".
+-08:02:21,892 nipype.workflow INFO:
+         [Node] Outdated cache found for "resting_preproc_sub-0009_ses-1.qc_skullstrip_1.montage_skull.montage_a".
+-08:02:22,252 nipype.workflow INFO:
+         [Node] Setting-up "_montage_a0" in "/scratch/working/resting_preproc_sub-0009_ses-1/qc_skullstrip_1/montage_skull/montage_a/mapflow/_montage_a0".
+-08:02:22,286 nipype.workflow INFO:
+         [Node] Outdated cache found for "_montage_a0".
+-08:02:22,446 nipype.workflow INFO:
+         [Node] Running "_montage_a0" ("CPAC.utils.function.Function")
+-08:02:28,560 nipype.workflow INFO:
+         [Node] Finished "_montage_a0".
+-08:02:28,576 nipype.workflow INFO:
+         [Node] Finished "resting_preproc_sub-0009_ses-1.qc_skullstrip_1.montage_skull.montage_a".
+srun: error: brick01: task 0: Exited with exit code 1
+</code>
+<code>
+nipype.external.portalocker.LockException: (5, 'Input/output error')
+-08:04:01,614 nipype.workflow INFO:
+         [Node] Setting-up "resting_preproc_sub-0001_ses-1.hist_vmhc_fisher_zstd_1" in "/scratch/working/resting_preproc_sub-0001_ses-1/_scan_rest_run-1/_selector_CSF-2mmE-M_aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1/_fwhm_4/hist_vmhc_fisher_zstd_1".
+-08:04:01,676 nipype.workflow INFO:
+         [Node] Outdated cache found for "resting_preproc_sub-0001_ses-1.hist_vmhc_fisher_zstd_1".
+-08:04:01,713 nipype.workflow INFO:
+         [Node] Running "hist_vmhc_fisher_zstd_1" ("CPAC.utils.function.Function")
+-08:04:01,899 nipype.workflow INFO:
+         [Node] Finished "resting_preproc_sub-0001_ses-1.hist_vmhc_fisher_zstd_1".
+srun: error: brick03: task 0: Exited with exit code 1
+</code>
+<code>
+         ***********************************
+-09:10:47,524 nipype.workflow ERROR:
+         could not run node: resting_preproc_sub-0014_ses-1.afni_centrality_0_eigenvector.afni_centrality.a0.b0
+-09:10:47,527 nipype.workflow INFO:
+         crashfile: /outputs/crash/crash-20190529-082639-osotolongo-afni_centrality.a0.b0-75dfa3ff-3c39-4800-b04d-c95080370a86.pklz
+-09:10:47,530 nipype.workflow ERROR:
+         could not run node: resting_preproc_sub-0014_ses-1.afni_centrality_1_eigenvector.afni_centrality.a0.b0
+-09:10:47,533 nipype.workflow INFO:
+         crashfile: /outputs/crash/crash-20190529-083301-osotolongo-afni_centrality.a0.b0-ee039f25-50ce-43c6-b64b-0d32355dd8c9.pklz
+-09:10:47,535 nipype.workflow INFO:
+         ***********************************
+</code>
+<code>
+nipype.external.portalocker.LockException: (5, 'Input/output error')
+-09:37:12,642 nipype.workflow INFO:
+         [Node] Running "_apply_ants_warp_mapnode6" ("nipype.interfaces.ants.resampling.ApplyTransforms"), a CommandLine Interface with command:
+antsApplyTransforms --default-value 0 --dimensionality 3 --float 0 --input /scratch/working/resting_preproc_sub-0010_ses-1/temporal_dual_regression_0/_scan_rest_run-1/_selector_CSF-2mmE-M_aC-CSF+WM-2mm-DPC5_G-M_M-SDB_P-2_BP-B0.01-T0.1/_spatial_map_PNAS_Smith09_rsn10_spatial_map_file_..cpac_templates..PNAS_Smith09_rsn10.nii.gz/split_raw_volumes/temp_reg_map_0006.nii.gz --input-image-type 0 --interpolation Linear --output temp_reg_map_0006_antswarp.nii.gz --reference-image /usr/share/fsl/5.0/data/standard/MNI152_T1_3mm_brain.nii.gz --transform /scratch/working/resting_preproc_sub-0010_ses-1/anat_mni_ants_register_0/calc_ants_warp/transform3Warp.nii.gz --transform /scratch/working/resting_preproc_sub-0010_ses-1/anat_mni_ants_register_0/calc_ants_warp/transform2Affine.mat --transform /scratch/working/resting_preproc_sub-0010_ses-1/anat_mni_ants_register_0/calc_ants_warp/transform1Rigid.mat --transform /scratch/working/resting_preproc_sub-0010_ses-1/anat_mni_ants_register_0/calc_ants_warp/transform0DerivedInitialMovingTranslation.mat --transform /scratch/working/resting_preproc_sub-0010_ses-1/dr_tempreg_maps_files_fsl_to_itk_0/_scan_rest_run-1/_selector_CSF-2mmE-M_aC-CSF+WM-2mm-DPC5_G-M_M-SDB_P-2_BP-B0.01-T0.1/_spatial_map_PNAS_Smith09_rsn10_spatial_map_file_..cpac_templates..PNAS_Smith09_rsn10.nii.gz/change_transform_type/mapflow/_change_transform_type6/updated_affine.txt
+-09:37:13,368 nipype.workflow INFO:
+         [Node] Finished "_apply_ants_warp_mapnode6".
+srun: error: brick01: task 0: Exited with exit code 1
+</code>
+===== Grupos =====
+Hay diferentes niveles de analisis grupal, explicados en http://fcp-indi.github.io/docs/user/group_analysis.html. Primero ha de elegirse lo que se desee hacer.