0

Trying out the queue system for a better user upload experience with Laravel-Excel.

.env was been changed from 'sync' to 'database' and migrations run. All the necessary use statements are in place yet the error above persists.

The exact error happens here:

Illuminate\Queue\Queue.php:97
        $payload = json_encode($this->createPayloadArray($job, $queue, $data));
                if (JSON_ERROR_NONE !== json_last_error()) {
                            throw new InvalidPayloadException(

If I drop ShouldQueue, the file imports perfectly in-session (large file so long wait period for user.)

I've read many stackoverflow, github etc comments on this but I don't have the technical skills to deep-dive to fix my particular situation (most of them speak of UTF-8 but I don't if that's an issue here; I changed the excel save format to UTF-8 but it didn't fix it.)

Ps. Whilst running the migration, I got the error:

SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes (SQL: alter table `jobs` add index `jobs_queue_index`(`queue`))

I bypassed by dropping the 'add index'; so my jobs table is not indexed on queue but I don't feel this is the cause.

user1729972
  • 404
  • 4
  • 18
  • I var_dumped and noticed $data is empty, $queue is a string with value "default" and $job is a huge object representing the chunked excel import. Searches indicate fiddling with utf8_encode, mb_convert_encoding and iconv but that would involve changing the framework code (and not clear how it won't mess things up, e.g. in the decoding process. – user1729972 Sep 02 '20 at 23:39

1 Answers1

1

One thing you can do when looking into json_encode() errors is use the json_last_error_msg() function, which will give you a bit more of a readable error message.

In your case you're getting a '5' back, which is the JSON_ERROR_UTF8 error code. The error message back for this is a slightly more informative one:

'Malformed UTF-8 characters, possibly incorrectly encoded'

So we know it's encountering non-UTF-8 characters, even though you're saving the file specifically with UTF-8 encoding. At first glance you might think you need to convert the encoding yourself in code (like this answer), but in this case, I don't think that'll help. For Laravel-Excel, this seems to be a limitation of trying to queue-read .xls files - from the Laravel-Excel docs:

You currently cannot queue xls imports. PhpSpreadsheet's Xls reader contains some non-utf8 characters, which makes it impossible to queue.

In this case you might be stuck with a slow, non-queueable option, or need to convert your spreadsheet into a queueable format e.g. .csv.

The key length error on running the migration is unrelated. It has been around for a while and is a side-effect of using an older version of MySQL/MariaDB. Check out this answer and the Laravel documentation around index lengths - you need to add this to your AppServiceProvider::boot() method:

Schema::defaultStringLength(191);
Leith
  • 2,800
  • 1
  • 24
  • 31
  • Had since abandoned laravel-excel to write through spout directly. My recollection originally failed me but in troubleshooting I actually converted to xlsx and the problem stayed same. – user1729972 Nov 19 '20 at 09:27
  • @user1729972 I think the 'Xls' reader applies to both xls and xlsx, likely it would need to be csv in order to be queueable. For CSV there's `league/csv` as well, but they don't seem to have an xls(x) package just yet. If you want to queue xlsx parsing, it might be more of a bespoke option only - or contribute your own modifications/packages to do it! :) – Leith Nov 20 '20 at 04:29