Stack Overflow
Stack Overflow

Questions tagged [lustre]

A highly scalable distributed file system.

Lustre is a highly scalable distributed file system. http://lustre.org/

36 questions
5
votes
1 answer

Unable to start name node while configuring Hadoop for Lustre

I'm trying to integrate hadoop with intel lustre. I have added hadoop-lustre-plugin-3.1.0 to hadoop-2.7.3/lib/native folder. Lustre is mounted at /mnt/lustre. I'm getting following error when I start hadoop using start-all.sh [root@master hadoop]#…
Trupti
  • 71
  • 2
4
votes
1 answer

Improve Fortran formatted I/O with a large number of small files

Lets assume I have the following requirements for writing monitor files from a simulation: A large number of individual files has to be written, typically in the order of 10000 The files must be human-readable, i.e. formatted I/O Periodically, a…
MechEng
  • 331
  • 1
  • 11
3
votes
2 answers

Mounting Lustre Inside Running Container Not Working (Have Added All Capabilities)

We are trying to mount lustre filesystem inside running container, and have successfully done this via containers which are running in priviledged mode. However for those containers which are running in non-privilidged mode, mounting lustre failed,…
Steve
  • 777
  • 2
  • 18
  • 42
3
votes
1 answer

C++ program is hanging when invoking 'mkfifo' (Lustre FS)

I'm running the C++ program STAR (https://github.com/alexdobin/STAR) which is quite common in my field of research. When I run this software on my server, everything goes fine. When I run the very same software on another server (FS=lustre) the…
Pierre
  • 31,741
  • 29
  • 101
  • 180
3
votes
1 answer

Can improved performance be obtained from parallel reading in Fortran?

I have a fortran90 code that spends (by far) most of the time on I/O, because very large data files (at least 1GB and up from there), need to be read. Smaller, but still large, data files with the results of the calculations need to be written.…
Miguel
  • 7,027
  • 1
  • 21
  • 40
2
votes
0 answers

Need test for asynchronous I/O

I am new to asynchronous I/O. I am trying to get it to work in C and Fortran codes on a Linux system. Based on what I can find out about the platform's somewhat outdated Lustre file system, the various aio routines are supported, after a fashion,…
bob.sacamento
  • 5,295
  • 7
  • 40
  • 95
2
votes
2 answers

What does ({;}) and ({0;}) in a macro definition signify?

I was going through the lustre source code and was stuck at the macro definition: #define ldlm_namespace_proc_unregister(ns) ({;}) #define ldlm_namespace_proc_register(ns) ({0;}) defined in the file lustre/ldlm/ldlm_resource.c. What…
Bhagyesh Dudhediya
  • 325
  • 2
  • 8
2
votes
1 answer

SBT forces file system locking, even on distributed file systems

I intended to run an extensive test suite that uses SBT on our university high performance computing cluster (it uses the Lustre file system). Since I have very basic user privileges, I was only able to try installing manually and installing by…
abanana
  • 140
  • 2
  • 10
2
votes
2 answers

What is scratch space /filesystem in HPC

I am studying about HPC applications and Parallel Filesystems. I came across the term scratch space AND scratch filesystem. I cannot visualize where this scratch space exists. Is it on the compute node as a mounted filesystem /scratch or on the…
RootPhoenix
  • 1,517
  • 20
  • 37
2
votes
1 answer

Running LevelDB on Lustre filesystem

I am running a piece of code that's based on LevelDB. It works fine in my workstation, but when I deploy it into a cluster (with Lustre file system), the program breaks with an "Invalid argument" error. This error is thrown by LevelDB. What I have…
user3570688
  • 21
  • 2
1
vote
1 answer

Need test to determine if asynchronous I/O is actually happening in C code

I am new to asynchronous I/O. I need to get it working in some C and Fortran programs on a Linux system. I managed to write a little C test code (included below) that reads asynchronously from two files. The code compiled and ran. What I am…
bob.sacamento
  • 5,295
  • 7
  • 40
  • 95
1
vote
1 answer

Searching a cluster filesystem (file storage) that is up-to-date and runs on both FreeBSD and Linux well

I have a FreeBSD 12.1-RELEASE server and a CentOS 7 server. Both run on amd64. I would like to set up a cluster file system, that runs on both platforms well. It should have CentOS 7 packages and FreeBSD packages. The solutions should be open-source…
Ronny Forberger
  • 157
  • 1
  • 14
1
vote
1 answer

How to install Lustre on AWS EC2 instances using Docker?

I need to deploy Lustre filesystem on AWS. I cannot use the Amazon FSx for Lustre but rather I need to create all the nodes running on EC2 instances. Also I need a docker configuration for this to simulate it locally. I imagine I need to use some…
Myhall
  • 71
  • 8
1
vote
1 answer

Lustre file locking for concurrent access

I'm trying to develop an application that will be running on multiple computers linked to a shared Lustre storage, performing various actions, including but not limited to: Appending data to a file. Reading data from a file. Reading from and…
Alceste_
  • 544
  • 3
  • 12
1
vote
2 answers

Error happens when trying to umount the Lustre file system

When I umount Lustre FS it displays: [root@cn17663-ens4 mnt]# umount /mnt/lustre umount: /mnt/lustre: target is busy. (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1)) and if I add the…
Li Hongbo
  • 11
  • 5
1
2 3