5

I'm trying to issue a LOAD DATA LOCAL INFILE query to load some CSV data into a table using the mysql2 gem (0.3.11) under rails 3.1.1:

class Foo < ActiveRecord::Base
  def self.load_csv
    query = "LOAD DATA LOCAL INFILE 'test/foo.csv' REPLACE INTO TABLE foos LINES TERMINATED BY '\n' (title)"
    ActiveRecord::Base.connection.execute(query)
  end
end

(This is a sample app to reproduce the error for this github issue). This keeps failing on OS X (Lion) with the following error:

Mysql2::Error: Malformed packet: LOAD DATA LOCAL INFILE 'test/foo.csv' REPLACE INTO TABLE foos LINES TERMINATED BY '
' (title)

Local infile is enabled on the server: 

mysql> show variables where variable_name like '%local%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| local_infile  | ON    |
+---------------+-------+

and on the client via this directive in application.rb:

Mysql2::Client.default_query_options[:connect_flags] |= Mysql2::Client::LOCAL_FILES

The same LOAD statement works fine from the MySQL client. Changing the DB connection method from socket to TCP/IP makes no difference. MySql is installed via homebrew and the version is

mysql  Ver 14.14 Distrib 5.5.15, for osx10.7 (i386) using readline 5.1

I do NOT get this error running the same code under Linux. It also works if I omit the LOCAL modifier, but that is not an option as the file is in fact local in production and the database server remote. It has nothing to do with file permissions as in this question.

This is driving me nuts, any insights are greatly appreciated.

Community
  • 1
  • 1
Thilo
  • 16,477
  • 5
  • 62
  • 80
  • How did you install MySQL on Lion? Through homebrew? – Romain Jan 09 '12 at 15:43
  • Yes - edited question accordingly – Thilo Jan 09 '12 at 15:44
  • Did you explicitly pass `--enable-local-infile` when doing `brew install mysql`? If not, could you try and see if the problem persists? – Romain Jan 09 '12 at 15:46
  • Tried that... makes no difference. – Thilo Jan 09 '12 at 16:20
  • From then on, we know it's a bug in the `mysql2` gem. I had the same and got to use the `mysql` gem instead, but it's band-aid on a broken arm, doesn't fix the root cause. I'm interested if anyone finds one, though. – Romain Jan 09 '12 at 16:32

2 Answers2

4

Issue should be fixed for mysql2 v > 0.3.12b4

It did not work for me using LOCAL_FILE flag, but adding :local_infile => true to options did the trick

1.9.3p194 :011 > a = Mysql2::Client.new(:username=>'root', :host=>'localhost', :password=>'', :database=>'bhl_indexer', :local_infile => true)

https://github.com/brianmario/mysql2/issues/293

Thilo
  • 16,477
  • 5
  • 62
  • 80
dimus
  • 6,970
  • 10
  • 38
  • 49
  • Either for me: `Mysql2::Client.default_query_options[:connect_flags] |= Mysql2::Client::LOCAL_FILES` causing 'Malformed packet error'. But adding :local_infile connection option worked. MySQL v.5.5.40, mysql2 gem v.0.3.16. – Grimmo Nov 05 '14 at 12:25
0

Turns out this is simply solved by re-installing the mysql2 gem. I think I had installed it with a MySQL version compiled from source, and then later switched to MySQL installed via brew, which led to an incompability in the client code used by the gem. That explains why it did work from the command line client but not when using the gem.

gem uninstall mysql2
gem install mysql2

Doh...

Thilo
  • 16,477
  • 5
  • 62
  • 80
  • I have Mysql version `5.5.19 Source distribution` installed via homebrew. I have upgraded to the latest mysql2 gem as of this writing, version `0.3.11`. I'm using the connection options specified above, but still getting the `Malformed packet (Mysql2::Error)` error when trying to use "load data" through the client in a script. I can run the same commands directly in the mysql client without the error. Any suggestions? – Andy Atkinson Feb 20 '12 at 03:16
  • Works ok on another OS X with Mysql `5.1.41 Source distribution` installed and on an Ubuntu box with Mysql `5.1.41-3ubuntu12.10 (Ubuntu)` installed. So I may roll-back my Mysql version as a workaround, since I can't figure out the "Malformed packet" error. – Andy Atkinson Feb 20 '12 at 06:02
  • There's really no reason do downvote an answer because it doesn't help your particular situation :) good that your rollback worked though. – Thilo Feb 20 '12 at 09:04
  • To install Mysql 5.1 on OS X with homebrew: http://stackoverflow.com/a/7470529/126688 @Thilo, re-installing the gem doesn't fix the issue. – Andy Atkinson Feb 21 '12 at 01:55
  • Ok finally got it. Using Mysql 5.1 installed with homebrew on OS X as specified in the previous comment, though I think the actual fix was specifying the ARCHFLAGS option when installing the mysql2 gem, and installing mysql2 gem version < 0.3. I installed as follows: `env ARCHFLAGS="-arch x86_64" gem install --no-rdoc --no-ri mysql2 -v '0.2.18' -- --with-mysql-config=/usr/local/bin/mysql_config`, then I updated my Gemfile with `gem "mysql2", "0.2.18"` and ran `bundle` which set the mysql2 gem version to the correct one. Now "load data" is working. – Andy Atkinson Feb 21 '12 at 02:23