Python for HPC

For batch processing, you have to write R script and Job script. R script can be written in R studio and saved as a script file. But to submit a job one needs to avail of PuTTY & WinSCP. A brief description of different steps are provided below (click on R for HPC for details).

  1. Download and install PuTTY from http://www.putty.org/ following the instructions therein.
  2. Download and install WinSCP from http://winscp.net/eng/docs/guide_installfollowing the instructions therein.
  3. Write a Python script test.py and job script test.slurm as given below

(a) Write the Python script file test.py given below

from mpi4py import MPI
import numpy
a = numpy.asarray([ [1,2,3], [4,5,6], [7,8,9] ])
numpy.savetxt("/datastore/kha04m/foo.csv", a, delimiter=",")

(b) Write the job file test.slurm given below

#!/bin/bash
#SBATCH --time=00:05:00
#SBATCH --nodes=1
#SBATCH --ntasks=3
#SBATCH --job-name=python_mpi
#SBATCH --mem=8000
module load python/2.7.10
module load openmpi/1.8.7
mpirun python ./test.py

(c) Submit the job on SLURM by using the command

sbatch test.slurm