Archive

Archive for April, 2016

CFD: Solicitud 2016 de licencias de STARCCM+ para investigación y docencia

April 13th, 2016

El Servicio General de Informática Aplicada a la Investigación (Cálculo Científico) ha renovado la licencia corporativa para la UPV/EHU del programa de dinámica de fluidos (CFD) STARCCM+ para el año 2016.

Estas licencias pueden usarse en los clusters de cálculo del Servicio de Cálculo Científico de la UPV/EHU. También pueden solicitarse licencias personales para uso en los ordenadores de los investigadores, para ello es necesario realizar una solicitud a los técnicos del Servicio de Cálculo Científico.

Características de la licencia

La licencia será válida hasta Febrero del 2017 y permite el uso de todos los procesadores del ordenador. Se puede usar en ordenadores corporativos de la UPV/EHU o a través de VPN.

Se dispone de licencia de docencia que permite la instalación de STARCCM+ en las aulas de docencia de la UPV/EHU para impartir cursos y asignaturas.

Tarifa

El uso de STARCCM+ en aulas para docencia de cursos y asignaturas es gratuito.

El uso de STARCCM+ en el cluster de cálculo Arina de la UPV/EHU es gratuito y solo se factura el uso de horas de cálculo según la tarifa estándar del Servicio.

Las tarifas que se aplicarán para la instalación de licencias en los ordenadores personales de los investigadores se detalla a continuación.

  • Licencias de docencia y pruebas.

Para un PC y menos de 250 horas. Se pueden solicitar licencias gratuitas para probar el programa, realizar tareas relacionadas con la docencia o proyectos de fin de máster. Estas licencias estarán restringidas a un máximo de 250 horas de uso.

  • Licencias de investigación en PCs

Las licencias destinadas a su instalación en ordenadores corporativos personales cuyo uso va a ser de más de 250 horas, y por tanto se consideran de investigación, se facturará a 140 € por equipo.

Se pueden solicitar acceso a equipos por VPN. Los recursos a traveś de VPN son muy limitados, por ello la licencia se renovará trimestralmente con un costo adicional de 35 € por licencia y trimestre.

  • Licencias de investigación en estaciones de trabajo y servidores.

Las licencias destinadas a su instalación en estaciones de trabajo y servidores (procesadores xeon o similares) tendrá un coste de 45 € por core.

STARCCM+ CFD simulationMás información

  • Página web CD-Adapco, proveedor del software.

.

Anuncios, IZO-SGI

Installing Tensorflow 0.7 in Red Hat Enterprise Linux Server 6.4 with GPUs

April 7th, 2016

Red Hat Enterprise Linux Server  6.4 has already a quite old OS, and it is not possible to install the precompiled tensorflow packages with pip and so on. So we had to compile it.

The instructions we follow are based on this document:

https://www.tensorflow.org/versions/r0.7/get_started/os_setup.html#installation-for-linux

We want to install it to be run on GPUs, so first we need to register in nvidia

https://developer.nvidia.com/cudnn

to install the cuDNN libraries. Download and copy the include and libraries to the corresponding cuda version directories.

In order to compile tensorflow from source, we first need to compile the bazel compiler.

1.- Installing Bazel

The first problem is that the gcc/g++ compiler in RHELS 6.4 is old,  4.4.7, and at least 4.8 is required. We did install RH devtoolset-3 which provides 4.9.2 version of gcc/g++. We also need Java JDK 8 or later.

Now, we can download bazel:

git clone https://github.com/bazelbuild/bazel.git

And we set up our environment to compile bazel:

export JAVA_HOME=/software/jdk1.8.0_20
export PATH=/opt/rh/devtoolset-3/root/usr/bin:/software/anaconda2/bin:/software/jdk1.8.0_20/bin:$PATH
export LD_LIBRARY_PATH=/opt/rh/devtoolset-3/root/usr/lib64:/opt/rh/devtoolset-3/root/usr/lib:/software/anaconda2/lib64:/software/anaconda2/lib:$LD_LIBRARY_PATH

Then, we have to modify the bazel/tools/cpp/CROSSTOOL file to choose the commands from devtoolset-3 instead of the default ones, in the toolchain_identifier: “local_linux”  

toolchain {
abi_version: “local”
abi_libc_version: “local”
builtin_sysroot: “”
compiler: “compiler”
host_system_name: “local”
needsPic: true
supports_gold_linker: false
supports_incremental_linker: false
supports_fission: false
supports_interface_shared_objects: false
supports_normalizing_ar: false
supports_start_end_lib: false
supports_thin_archives: false
target_libc: “local”
target_cpu: “local”
target_system_name: “local”
toolchain_identifier: “local_linux”
tool_path { name: “ar” path: “/opt/rh/devtoolset-3/root/usr/bin/ar” }
tool_path { name: “compat-ld” path: “/opt/rh/devtoolset-3/root/usr/bin/ld” }
tool_path { name: “cpp” path: “/opt/rh/devtoolset-3/root/usr/bin/cpp” }
tool_path { name: “dwp” path: “/opt/rh/devtoolset-3/root/usr/bin/dwp” }
tool_path { name: “gcc” path: “/opt/rh/devtoolset-3/root/usr/bin/gcc” }
cxx_flag: “-std=c++0x”
linker_flag: “-lstdc++”
linker_flag: “-B/opt/rh/devtoolset-3/root/usr/bin/

# TODO(bazel-team): In theory, the path here ought to exactly match the path
# used by gcc. That works because bazel currently doesn’t track files at
# absolute locations and has no remote execution, yet. However, this will need
# to be fixed, maybe with auto-detection?
cxx_builtin_include_directory: “/opt/rh/devtoolset-3/root/usr/lib/gcc/
cxx_builtin_include_directory: “/opt/rh/devtoolset-3/root/usr/include
tool_path { name: “gcov” path: “/opt/rh/devtoolset-3/root/usr/bin/gcov” }

# C(++) compiles invoke the compiler (as that is the one knowing where
# to find libraries), but we provide LD so other rules can invoke the linker.
tool_path { name: “ld” path: “/opt/rh/devtoolset-3/root/usr/bin/ld” }

tool_path { name: “nm” path: “/opt/rh/devtoolset-3/root/usr/bin/nm” }
tool_path { name: “objcopy” path: “/opt/rh/devtoolset-3/root/usr/bin/objcopy” }
objcopy_embed_flag: “-I”
objcopy_embed_flag: “binary”
tool_path { name: “objdump” path: “/opt/rh/devtoolset-3/root/usr/bin/objdump” }
tool_path { name: “strip” path: “/opt/rh/devtoolset-3/root/usr/bin/strip” }
compilation_mode_flags {
mode: DBG
# Enable debug symbols.
compiler_flag: “-g”
}
compilation_mode_flags {
mode: OPT
# No debug symbols.
# Maybe we should enable https://gcc.gnu.org/wiki/DebugFission for opt or even generally?
# However, that can’t happen here, as it requires special handling in Bazel.
compiler_flag: “-g0”

# Conservative choice for -O
# -O3 can increase binary size and even slow down the resulting binaries.
# Profile first and / or use FDO if you need better performance than this.
compiler_flag: “-O2”

# Disable assertions
compiler_flag: “-DNDEBUG”

# Removal of unused code and data at link time (can this increase binary size in some cases?).
compiler_flag: “-ffunction-sections”
compiler_flag: “-fdata-sections”
}
linking_mode_flags { mode: DYNAMIC }
}

Now, we can compile it with the command:

./compile.sh

It will create a binary bazel, that we will use now to compile tensorflow.

 

2.- Tensorflow

Download Tensorflow

git clone –recurse-submodules https://github.com/tensorflow/tensorflow

We define the environment to compile tensorflow with the devtools-3 and with cuda

export JAVA_HOME=/software/jdk1.8.0_20

export PATH=/software/jdk1.8.0_20/bin:/opt/rh/devtoolset-3/root/usr/bin:/software/anaconda2/bin:/software/cuda-7.5.18/bin:$PATH

export LD_LIBRARY_PATH=/opt/rh/devtoolset-3/root/usr/lib64:/opt/rh/devtoolset-3/root/usr/lib:/software/cuda-7.5.18/lib64:/software/anaconda2/lib64:/software/anaconda2/lib:$LD_LIBRARY_PATH

We run configure in tensorflow to setup our cuda envirnment:

cd tensorflow

Fix the google/protobuf/BUILD file changing:

LINK_OPTS = [“-lpthread”]

to

LINK_OPTS = [“-lpthread”,”-lrt”,”-lm”]

and configure it

./configure
Please specify the location of python. [Default is /software/anaconda2/bin/python]:
Do you wish to build TensorFlow with GPU support? [y/N] y
GPU support will be enabled for TensorFlow
Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 7.5
Please specify the location where CUDA 7.5 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /software/cuda-7.5.18
Please specify the Cudnn version you want to use. [Leave empty to use system default]:
Please specify the location where cuDNN library is installed. Refer to README.md for more details. [Default is /software/cuda-7.5.18]:
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size.
[Default is: “3.5,5.2”]: 3.5

 

As we did in bazel, we need to fix the CROSSTOOL file third_party/gpus/crosstool/CROSSTOOL:

 

toolchain {
abi_version: “local”
abi_libc_version: “local”
builtin_sysroot: “”
compiler: “compiler”
host_system_name: “local”
needsPic: true
supports_gold_linker: false
supports_incremental_linker: false
supports_fission: false
supports_interface_shared_objects: false
supports_normalizing_ar: false
supports_start_end_lib: false
supports_thin_archives: false
target_libc: “local”
target_cpu: “local”
target_system_name: “local”
toolchain_identifier: “local_linux”

tool_path { name: “ar” path: “/opt/rh/devtoolset-3/root/usr/bin/ar” }
tool_path { name: “compat-ld” path: “/opt/rh/devtoolset-3/root/usr/bin/ld” }
tool_path { name: “cpp” path: “/opt/rh/devtoolset-3/root/usr/bin/cpp” }
tool_path { name: “dwp” path: “/opt/rh/devtoolset-3/root/usr/bin/dwp” }
# As part of the TensorFlow release, we place some cuda-related compilation
# files in third_party/gpus/crosstool/clang/bin, and this relative
# path, combined with the rest of our Bazel configuration causes our
# compilation to use those files.
tool_path { name: “gcc” path: “clang/bin/crosstool_wrapper_driver_is_not_gcc” }
# Use “-std=c++11” for nvcc. For consistency, force both the host compiler
# and the device compiler to use “-std=c++11”.
cxx_flag: “-std=c++11”
linker_flag: “-lstdc++”
linker_flag: “-B/opt/rh/devtoolset-3/root/usr/bin/”

# TODO(bazel-team): In theory, the path here ought to exactly match the path
# used by gcc. That works because bazel currently doesn’t track files at
# absolute locations and has no remote execution, yet. However, this will need
# to be fixed, maybe with auto-detection?
cxx_builtin_include_directory: “/opt/rh/devtoolset-3/root/usr/lib/gcc/”
cxx_builtin_include_directory: “/usr/local/include”
cxx_builtin_include_directory: “/usr/include”
cxx_builtin_include_directory: “/opt/rh/devtoolset-3/root/usr/include”
tool_path { name: “gcov” path: “/opt/rh/devtoolset-3/root/usr/bin/gcov” }

# C(++) compiles invoke the compiler (as that is the one knowing where
# to find libraries), but we provide LD so other rules can invoke the linker.
tool_path { name: “ld” path: “/opt/rh/devtoolset-3/root/usr/bin/ld” }

tool_path { name: “nm” path: “/opt/rh/devtoolset-3/root/usr/bin/nm” }
tool_path { name: “objcopy” path: “/opt/rh/devtoolset-3/root/usr/bin/objcopy” }
objcopy_embed_flag: “-I”
objcopy_embed_flag: “binary”
tool_path { name: “objdump” path: “/opt/rh/devtoolset-3/root/usr/bin/objdump” }
tool_path { name: “strip” path: “/opt/rh/devtoolset-3/root/usr/bin/strip” }

# Anticipated future default.
unfiltered_cxx_flag: “-no-canonical-prefixes”

# Make C++ compilation deterministic. Use linkstamping instead of these
# compiler symbols.
unfiltered_cxx_flag: “-Wno-builtin-macro-redefined”
unfiltered_cxx_flag: “-D__DATE__=\”redacted\””
unfiltered_cxx_flag: “-D__TIMESTAMP__=\”redacted\””
unfiltered_cxx_flag: “-D__TIME__=\”redacted\””

# Security hardening on by default.
# Conservative choice; -D_FORTIFY_SOURCE=2 may be unsafe in some cases.
# We need to undef it before redefining it as some distributions now have
# it enabled by default.
compiler_flag: “-U_FORTIFY_SOURCE”
compiler_flag: “-D_FORTIFY_SOURCE=1”
compiler_flag: “-fstack-protector”
compiler_flag: “-fPIE”
linker_flag: “-pie”
linker_flag: “-Wl,-z,relro,-z,now”

# Enable coloring even if there’s no attached terminal. Bazel removes the
# escape sequences if –nocolor is specified. This isn’t supported by gcc
# on Ubuntu 14.04.
# compiler_flag: “-fcolor-diagnostics”

# All warnings are enabled. Maybe enable -Werror as well?
compiler_flag: “-Wall”
# Enable a few more warnings that aren’t part of -Wall.
compiler_flag: “-Wunused-but-set-parameter”
# But disable some that are problematic.
compiler_flag: “-Wno-free-nonheap-object” # has false positives

# Keep stack frames for debugging, even in opt mode.
compiler_flag: “-fno-omit-frame-pointer”

# Anticipated future default.
linker_flag: “-no-canonical-prefixes”
unfiltered_cxx_flag: “-fno-canonical-system-headers”
# Have gcc return the exit code from ld.
linker_flag: “-pass-exit-codes”
# Stamp the binary with a unique identifier.
linker_flag: “-Wl,–build-id=md5”
linker_flag: “-Wl,–hash-style=gnu”
# Gold linker only? Can we enable this by default?
# linker_flag: “-Wl,–warn-execstack”
# linker_flag: “-Wl,–detect-odr-violations”

compilation_mode_flags {
mode: DBG
# Enable debug symbols.
compiler_flag: “-g”
}

Similarly, we also need to fix /third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc to choose the devtools-3 tools.

Now, we will build a build_pip_package:

bazel build -c opt –config=cuda –genrule_strategy=standalone –verbose_failures //tensorflow/tools/pip_package:build_pip_package

I got an error, "ImportError: No module named argparse" so I had to change also the first line of /third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc to:

#!/usr/bin/env /software/anaconda2/bin/python2

Then, we create the python wheel package:

tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

In my case, I had to run

/root/.cache/bazel/_bazel_root/28150a65056607dabfb056aa305868ed/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

And finally we can install it using pip:

bazel build -c opt –config=cuda –genrule_strategy=standalone //tensorflow/cc:tutorials_example_trainer

pip install /tmp/tensorflow_pkg/tensorflow-0.7.1-py2-none-any.whl

We also tried to compile the tutorials_example_trainer as shown in the tensorflow webpage:

bazel build -c opt –config=cuda –genrule_strategy=standalone //tensorflow/cc:tutorials_example_trainer

And successfully run the test:

tutorials_example_trainer –use_gpu

 

We would like to thank Kike from IXA-taldea for sharing with us his guide to compile tensorflow in CPUs.

 

 

General, HPC