0

I am working on a project that requires me to download images into sub directories. Instead of writing my own code, I elected to use code that is already available for the sake of time. However, I'm running into a slight problem with creating directory if directory does not exist.

After some extensive troubleshooting, I notice the images are not saving in the appropriate folders. This is happening even if the folders exist in the path. In short, the images are not saving in the nested directory.

Is is possible to create sequential nested directories with os.makedirs?

for example: 

kits19process\Image\0\img1.png
kits19process\Image\1\img1.png
kits19process\Image\2\img1.png

I also tried using os.makedirs(tumorimagefile, exist_ok=True) as well. But that technique raised an error.

I have hunch that the problem lies here. I may be wrong, so here is the link to the full script.


        trainimagefile = proImage + str(seriesindex)
        trainMaskfile = proMask + str(seriesindex)
        if not os.path.exists(trainimagefile):
            os.makedirs(trainimagefile)
        if not os.path.exists(trainMaskfile):
            os.makedirs(trainMaskfile)

        tumorimagefile = protumorImage + str(seriesindex)
        tumorMaskfile = protumorMask + str(seriesindex)
        if not os.path.exists(tumorimagefile):
            os.makedirs(tumorimagefile)
        if not os.path.exists(tumorMaskfile):
            os.makedirs(tumorMaskfile

Also here is a screenshot of what is happening when I run this code (incorrect way).

Not the desire results]([![https://user-images.githubusercontent.com/25371067/56928885-c04a1100-6a9d-11e9-851f-c456d39ae322.png]1).

As you can see the files are not saving into the correct folders.

I expect the actual output is to have the images and mask saved in the appropriate folders. 

data/
    Kits19/
          kits19process/
                      Image/
                           0/
                           0.bmp
                      Mask/
                           0/
                           0.bmp
         kits19tumorprocess/
                      Image/
                           0/
                           0.bmp

                      Mask/
                           0/
                           0.bmp
Green
  • 567
  • 2
  • 7
  • 18
  • https://stackoverflow.com/questions/273192/how-can-i-safely-create-a-nested-directory-in-python https://stackoverflow.com/questions/600268/mkdir-p-functionality-in-python – lampslave Apr 30 '19 at 18:30
  • Just to be absolutely clear with what's going on, can you use either a `print()` or a debugger to verify the values of `trainimagefile`, `trainMaskfile`, etc. before you call `mkdirs()` on them? I'm not a Windows person, but it's probably possible that they aren't formatted correctly as directories (in which case the surest solution would be to refactor the whole code to use `os.path.join()` instead of string concatenation when making filenames) – Green Cloak Guy Apr 30 '19 at 18:34
  • Use raw strings when writing pathnames with backslashes in them, or use forward slashes instead of backslashes. – Barmar Apr 30 '19 at 19:23
  • The script you linked to uses `E:`, why does your screenshot here show `D:`? – Barmar Apr 30 '19 at 19:24
  • @Barmar I don't think the drive location matters. I'm actually using Floydhub. The point I was making is that regardless of directory path, the files are not saving correctly or in the correct folders. – Green Apr 30 '19 at 19:44
  • The problem is that the code you linked to is obviously not the same code you're having trouble with. The drive letters aren't the only differences in the pathnames. – Barmar Apr 30 '19 at 19:46
  • @Barmar The screen show's ```D:``` because the drive path was updated. My drive path is different from the author's path. Rather it's ```E:``` or ```D:```, the drive location should not matter. For example if you are using cloud, than obviously the drive path is going to be different. It's going to be in according to your environment. – Green May 01 '19 at 15:13
  • Who is "the author"? I thought that was your script. The exact path matters, because you have backslashes in the pathnames. And since you didn't use raw strings, some of them will be treated as escape sequences rather than directory separators. So try again with raw strings or forward slashes. – Barmar May 01 '19 at 15:22
  • @Barmar I inserted the link with the problem description above. I tried the both backslashes and forward slashes and ended up with the same results. I even created all the folders and subfolders. All the images saved outside their designated folder. I thought that was kind of odd. Also, I believe the intention of ```if not os.path.exists``` is to check if path exists and if not, it creates the path. When you run the code, it kind of does the opposite. It checks if the path exists, if path doesn't exist, It creates the folder without a subfolder. So it is not doing what it's design to do. – Green May 01 '19 at 16:22
  • @Barmar It's not creating a nested directory if path doesn't exist. Essentially it should create series of folders with images and masks saved inside the folder. Are you saying this portion of the code needs to be change ```trainimagefile = proImage + str(seriesindex)```? – Green May 01 '19 at 16:25
  • The data is saving into `D:\Data\KiTS\kits19process\Image\0\0.bmp`, but it should be in `kits19process\Image\0\0.png`. What's the problem there? Is it the `\Data\kITS` at the beginning? Is it because it's in `.bmp` instead of `.png`? – Barmar May 01 '19 at 17:58
  • The question starts out asking about sequentially numbered directories, but at the end you say the expected result should all be in directories named `0`. – Barmar May 01 '19 at 18:00
  • I'm struggling to see which nested directories it's not creating. Try simplifying the question, and focus on just one directory that should have been created and wasn't. – Barmar May 01 '19 at 18:02
  • That's the 4th line from the bottom in the screenshot. – Barmar May 01 '19 at 19:37
  • @Barmar Looking at the screen shot you can see two things. The images are not being name correctly and not stored in the appropriate folder. You see that folders are being created sequentially. But the images are not being name correctly. Nor are they being saved in the correct folders. – Green May 01 '19 at 19:50
  • @Barmar When I saw the error, I ran a code that I created to make a nested directory versus manually creating 100s of folders. I done that to ensure that the paths are there. I re-ran the code and I got similar results. It created the images and did not save them in the appropriate folder. In fact it saved the images here: ```D:\Data\KiTS\kits19process```. So now you have images that resides outside the ```Image``` folder. – Green May 01 '19 at 19:55
  • IAt the end of the question you say "I expect the actual output to be" and then all 3 lines are in the same folder. That folder is the same as the last 4 lines of the screen shot, isn't it? The image names being `.bmp` instead of `.png` has nothing to do with the folders, that's a different problem. – Barmar May 01 '19 at 20:03
  • You're getting `.bmp` filenames because of `cv2.imwrite(trainimagefile + "\\" + str(z) + ".bmp", src_kineryimg[z])` – Barmar May 01 '19 at 20:04
  • I suggest you add `print()` statements to the script so you can see what the directory names are when it's trying to create them. – Barmar May 01 '19 at 20:05
  • @Barmar I'm fine with getting ```.bmp```. My mistake. The actual output should be ```kits19process\Image\0\0.bmp kits19process\Image\0\1.bmp kits19process\Image\0\2.bmp kits19process\Image\0\0.bmp kits19process\Image\1\4.bmp kits19process\Image\2\1.bmp``` – Green May 01 '19 at 20:11
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/192687/discussion-between-barmar-and-hello-world). – Barmar May 01 '19 at 20:13

1 Answers1

0

@Barmar Thank you for your help and patience!

I found were the problem lies. 

def gen_subregion(srcimg, seg_image, trainimagefile, trainMaskfile, expandslice=5):
    # 5 get the roi range of mask,and expand number slices before and after,and get expand range roi image
    # for kinery
    startpostion, endpostion = getRangImageDepth(seg_image)
    if startpostion == endpostion:
        return
    imagez = np.shape(seg_image)[0]
    startpostion = startpostion - expandslice
    endpostion = endpostion + expandslice
    if startpostion < 0:
        startpostion = 0
    if endpostion > imagez:
        endpostion = imagez

    src_kineryimg = srcimg[startpostion:endpostion, :, :]
    seg_kineryimage = seg_image[startpostion:endpostion, :, :]
    # 6 write src, liver mask and tumor mask image
    for z in range(seg_kineryimage.shape[0]):
        src_kineryimg = np.clip(src_kineryimg, 0, 255).astype('uint8')
        cv2.imwrite(trainimagefile + "//" + str(z) + ".bmp", src_kineryimg[z])
        cv2.imwrite(trainMaskfile + "//" + str(z) + ".bmp", seg_kineryimage[z])

I had to change \\ to //.

Green
  • 567
  • 2
  • 7
  • 18