4

I have scala application with akka steams. So the flow of my application is like this:

1. Check if file exists on FTP - I'm doing it with the org.apache.commons.net.ftp.FTPClient
2. If it exists stream it via alpakka library(and make some stream transformations)

My application works locally and it can connect to the server.

The problem is when it is being deployed to dcos/mesos. I get an issue:

java.io.IOException: /path/file.txt: No such file or directory

I can say for sure that file still exists there. Also when I try to connect from docker container locally through the ftp I've got something like this: 

ftp> open some.ftp.address.com
Connected to some.ftp.address.com.
220 Microsoft FTP Service
Name (some.ftp.address.com:root): USER
331 Password required
Password:
230 User logged in.
Remote system type is Windows_NT.
ftp> dir
501 Server cannot accept argument.
ftp: bind: Address already in use
ftp>
fr3ak
  • 329
  • 2
  • 14

2 Answers2

1

So my problem was really weird. But I've managed to fix this way. Quick answer: I was using alpakka ftp lib this way:

Ftp
  .fromPath(url, user, pass, Paths.get(s"/path/$fileName"))

But using this way it works:

val ftpSettings = FtpSettings(
  host = InetAddress.getByName(url),
  port = 21,
  NonAnonFtpCredentials(user, pass),
  binary = true,
  passiveMode = true
)
Ftp
  .fromPath(Paths.get(s"/path/$fileName"), ftpSettings)

Longer answer: I started investigating alpakka lib and I've discovered that it uses the same lib that works for me during checking if file exists!

https://github.com/akka/alpakka/blob/master/ftp/src/main/scala/akka/stream/alpakka/ftp/impl/FtpOperations.scala

So I've started digging and it seems that most likely tahat setting passive mode to true was the solution. But it's weird because I've read that windows ftp server does not support passive mode... I hope someone could clarify my doubts one day, but at the moment I'm happy because it works :)

fr3ak
  • 329
  • 2
  • 14
  • Does the solution mean that the problem had nothing to do with DCOS? – Jacek Laskowski Feb 01 '17 at 20:34
  • really it had to do with the docker container, that it could not connect from inside of it, but yet I still may be wrong, because I've found the fix not really found the reason why is this happening – fr3ak Feb 02 '17 at 06:03
  • Adding passiveMode = true solved also for me inside a docker. Thanks. – Dario Balinzo Feb 21 '18 at 11:54
1

Not sure if its still helpful but I also got my ftp client transfering data from inside a Docker container after changing the data connection to passive. I think that active mode requires the client to have open ports which the server connects to when returning file listing results and during data transfer. However the client ports are not reachable from outside of the docker container since the requests are not routed through (like in a NAT:et network).

Found this post explaning active/passive FTP connections

https://labs.daemon.com.au/t/active-vs-passive-ftp/182

Andreas
  • 146
  • 1
  • 4