Nice explanation of the basic parallel programing principles and paradigms. https://hpc.llnl.gov/training/tutorials The following ones cover most of the topic. https://computing.llnl.gov/tutorials/parallel_comp/ https://computing.llnl.gov/tutorials/linux_clusters/ https://computing.llnl.gov/tutorials/pthreads/ https://computing.llnl.gov/tutorials/openMP/ https://computing.llnl.gov/tutorials/mpi/ Parallel Programming Learning Materials