How to read from RAM?

Question

What I need to do?

I need to read the file from RAM

How I am doing it?

I use std::istream, because it know how to read from buffer. So, there is method that know how to read file size(according to this SO answer https://stackoverflow.com/a/6039648/5709159)

long getFileSize(const std::string &filename)
{
    struct stat stat_buf{};
    int rc = stat(filename.c_str(), &stat_buf);
    return rc == 0 ? stat_buf.st_size : -1;
}

Then I use membuf to copy bytes from file to buffer (according to this SO answer https://stackoverflow.com/a/13586356/5709159)

struct membuf : std::streambuf
    {
        membuf(char *base, std::ptrdiff_t n)
        {
            this->setg(base, base, base + n);
        }
    };

Full implementation looks like this

long size = getFileSize(filename);
membuf sbuf(0, size);
std::istream file(&sbuf);

But, I am now sure that sbuf(0, size); that 0 here is on the right place...

So, question is - how to set membuf with right size ?

@Slava why? I just would like to know if set membuf like I do it, is ok or not? — Aleksey Timoshchenko, Jul 24 '19 at 14:17
Because it means something completely different than question itself if it means anything. Did you use online translator? — Slava, Jul 24 '19 at 14:25
@Slava no, but as well I am not a native speaker... I changed the title of question... — Aleksey Timoshchenko, Jul 24 '19 at 14:39
First argument is start of the buffer. You set to 0 which means it will try to read from zero address in RAM. Is that what you want? On most operating systems it will cause a crash. — Ilya Popov, Jul 24 '19 at 14:55
Also not clear what are you trying to achieve? Are you trying to create an object of `std::istream` in such a way as to read from memory instead of a file? — Ilya Popov, Jul 24 '19 at 14:56
@IlyaPopov Actually I want to copy bytes from file to RAM (as a buffer) and read from RAM instead of read from file (at least it should work if I understand it correctly). — Aleksey Timoshchenko, Jul 24 '19 at 15:02
@IlyaPopov What do you mean alternative? As far as I know isringstream not read from RAM, no? — Aleksey Timoshchenko, Jul 24 '19 at 15:03
@IlyaPopov it is exactly what this guy is talking about https://stackoverflow.com/a/13586356/5709159 — Aleksey Timoshchenko, Jul 24 '19 at 19:14
Note that `long getFileSize(const std::string &filename)` is not right. The `st_size` field of a `struct stat` is of type `off_t`. A `long` value is not guaranteed to be large enough to hold an `off_t` value. For example, a 32-bit system that supports files larger than 2 GB will have to have an `off_t` that's larger than the 32-bit `long` type typical on 32-bit systems. Such a system likely has a 64-bit `off_t`. And then there's Windows, where `long` is 32-bits even for a 64-bit process... — Andrew Henle, Jul 24 '19 at 21:41
Regarding your question, are you trying to copy data from the file into RAM, or from a known location in RAM into the file? Or, are you trying to treat a known area of memory as a file and read from it as if that memory were a file? If that last, this might be helpful: https://stackoverflow.com/questions/29849749/what-is-the-difference-between-fmemopen-and-open-memstream — Andrew Henle, Jul 24 '19 at 21:46
@AndrewHenle So, if I understood you correctly you mean that this SO answer is not correct `about getSize()` https://stackoverflow.com/a/6039648/5709159 , right? — Aleksey Timoshchenko, Jul 25 '19 at 04:38
@AndrewHenle actually I would like to do exactly the same thing like write here https://stackoverflow.com/a/13586356/5709159 , but I don't understand what is `base` here... — Aleksey Timoshchenko, Jul 25 '19 at 04:40
@AlekseyTimoshchenko `base` is just starting address of the buffer from where your `istream` will be reading. You have to allocate this buffer yourself (for example, with `char *buffer = new char[size];`) and fill it with data. — Ilya Popov, Jul 25 '19 at 07:46
@AlekseyTimoshchenko `istringstream` reads from a string. And string is in RAM. If you put your data in a string, you can use `istringstream` to read the data from RAM. — Ilya Popov, Jul 25 '19 at 07:48

score 1 · Accepted Answer · answered Jul 25 '19 at 08:06

Option 1

Expanding on the code in your question:

long size = getFileSize(filename);
std::vector<char> buffer(size);
// here fill the buffer with data
membuf sbuf(buffer.data(), buffer.size());
std::istream file_from_ram(&sbuf);

if you'd like to fill the buffer with contenst of some file, do this:

std::istream original_file("filename.dat");
original_file.read(buffer.data(), size);

Note: this method has problems. First, as @andrew-henle mentioned in comments, file size of type off_t which may not fil into long. Second, this method suffers from TOCTOU (time of check -- time of use) problem: if file size changes between the call to getFileSize and reading the file, you may be in trouble.

Option 2

Use std::istringstream instead:

std::string buffer;
// here fill the string with data
std::istringstream file_from_ram(buffer);

If you'd like to fill the buffer with contents of some file, do this (from https://stackoverflow.com/a/116220/4451432):

std::ifstream original_file("filename.dat");

std::istringstream file_from_ram;
file_from_ram << original_file.rdbuf();
// Now you can read from file_from_ram

Actually first option that you provided I have already done, but it was intresting to check execution time about second one, and it is takes `*2` more time... I hope it is due to method every call to method make memory allocation for `std::string` where as `std::vector` I declered in `.h` file and just resize it... So, first option faster (as I tested). And also I get error hire `file_from_ram << original_file.rdbuf();` and after I changed `std::istringstream` to `std::stringstream` it works. — Aleksey Timoshchenko, Jul 25 '19 at 08:35

score 0 · Answer 2 · answered Jul 24 '19 at 15:48

0

As far as I understand your question, you want to read a file.

You need to understand that every variable of your program is stored in your ram. So to "read a file from ram" you just read your file as you are used to.

If the problem is about storing the file in your ram to use it later in another program, it is just not correct. Read the file from the program which need it.

answered Jul 24 '19 at 15:48

Tiphaine

193
9

So, how could you explain this answer https://stackoverflow.com/a/13586356/5709159 ? – Aleksey Timoshchenko Jul 24 '19 at 19:13
The answer you sent is about efficiency. Since an entire file can be stored as a C style char pointer, it will obviously take less memory than storing the std::istream class. If you really want to do it this way you should take a look at [fopen](http://man7.org/linux/man-pages/man3/fopen.3.html) and [getline](http://man7.org/linux/man-pages/man3/getline.3.html) – Tiphaine Jul 25 '19 at 08:00

How to read from RAM?

2 Answers2

Option 1

Option 2