<tt>diststream</tt>

i/o streams in distributed environment

Description

The idiststream and odiststream classes provide a stream input and output interface for parallel and distributed codes. The main difference Recall that, with the usual std::istream and std::ostream, any i/o operation are executed on all processes: e.g. the output is printed many times. In contrast, idiststream and odiststream manage nicely the distributed environment.

For small data, e.g. int, double or string, a specific processor is selected for i/o operations. For large data, the i/o operations are delegated to a specific class member functions when available.

For convenience, the standard streams cin, cout, clog and cerr are extended as din, dout, dlog and derr, respectively.

File suffixes

Finally, optional suffix extension to the file name is automatically handled:

    odiststream foo("NAME", "suffix");

is similar

    ofstream foo("NAME.suffix").

Conversely,

    irheostream foo("NAME","suffix");

is similar to

    ifstream foo("NAME.suffix").

File search

Recursive search in a directory list is provided for finding the input file. However, we look at a search path environment variable RHEOPATH in order to find NAME while the suffix is assumed. Moreover, gzip compressed files, ending with the .gz suffix are handled, and decompression is done automatically on the fly in that case.

Compression

File compression/decompresion is handled on the fly, thanks to the gzip library. The data compression is assumed by default for output: it can be deactivated while opening a file by an optional argument:

    odiststream foo("NAME", "suffix", io::nogz);

Append mode

An existing file, possibly compressed, can be reopen in append mode. New results will be appended at the end of an existing file:

    odiststream foo("NAME", "suffix", io::app);

Flush

The flush member function is nicely handled in compression mode: this feature allows intermediate results to be available during long computations. Buffers are flushed and data are available in the output file.

Implementation

This documentation has been generated from file linalg/lib/diststream.h

class idiststream {
public:
  typedef std::size_t size_type;
 
// allocators/deallocators:
     
  idiststream();
  idiststream (std::istream& is, const communicator& comm = communicator());
  idiststream (std::string filename, std::string suffix = "",
             const communicator& comm = communicator());
  ~idiststream();
 
// modifiers:
 
  void open (std::string filename, std::string suffix = "",
             const communicator& comm = communicator());
  void close();
 
// accessors:
 
  const communicator& comm() const { return _comm; }
  bool good() const;
  operator bool() const { return good(); }
  static size_type io_proc();

};

class odiststream {
public:
  typedef std::size_t size_type;
 
// allocators/deallocators:
 
  odiststream();
  odiststream (std::string filename, std::string suffix = "",
              io::mode_type mode = io::out, const communicator& comm = communicator());
  odiststream (std::string filename,
              io::mode_type mode, const communicator& comm = communicator());
  odiststream (std::string filename, std::string suffix, const communicator& comm);
  odiststream (std::string filename, const communicator& comm);
  odiststream(std::ostream& os, const communicator& comm = communicator());
  ~odiststream();
 
// modifiers:
 
   void open (std::string filename, std::string suffix = "",
             io::mode_type mode = io::out, const communicator& comm = communicator());
   void open (std::string filename, 
             io::mode_type mode, const communicator& comm = communicator());
   void open (std::string filename, std::string suffix,
             const communicator& comm);
   void open (std::string filename, const communicator& comm);
   void flush();
   void close();
 
// accessors:
 
   const communicator& comm() const { return _comm; }
   bool good() const;
   operator bool() const { return good(); }
   static size_type io_proc();

};