How can I send POST requests from a background service in Java?

Question

I'm currently working on a Java application that reads from Table A (with BLOB stored), writes some data from table A to table B while uploading the BLOB data to a file server. I tested the application on a test database(with around 400 rows) and it works fine. I need to implement the application as a background service that reads table A and sends HTTP POST requests to a REST server, followed by an insertion to table B and upload to file server. After the POST request, the server needs to return HTTP 202 created. I tried something like this:

@POST
@Path("attachments")
public void moveToMinio() throws Exception {
    TiedostoDaoImpl tiedostoDao = new TiedostoDaoImpl();
    List<Integer> id = tiedostoDao.getDistinctCustomerId();
    for (Integer userId : id){
    AttachmentService.insertAndUploadService(userId);
    }
}

tiedostoDao.getDistinctCustomerId() returns a list of distinct customer id from table A and passes that id to AttachmentService.insertAndUploadService() inside a for loop. This somehow gets the job done, but I doubt this isn't the right way as it returns HTTP 200 and not 202. Is this right way to send a POST request? The production database may have millions of rows, what's the right way to process all those rows without affecting the server efficiency? I've been stuck with this for a while as I'm a java newbie and would really appreciate any help/suggestion.

What is your ultimate question? Returning 202? Having max. efficiency? Or any other thing I miss in the question? — vahdet, Jul 04 '18 at 07:04
@vahdet my ultimate question is how can I implement my application as a background service that sends POST requests to the server and returns an HTTP 202 response. — Roshan Upreti, Jul 04 '18 at 07:08
Do you just need the method moveToMinio to return 202 response? — Azarea, Jul 04 '18 at 07:16
@Azarea Yes. When the POST request is sent to /attachments, it starts to process the rows. There could be millions of rows and for every row processed it should return an HTTP 202 accepted with an upload link from the file server. — Roshan Upreti, Jul 04 '18 at 07:21
@RoshanUpreti but this method is a sync one, in this case only after all the userid are inserted and uploaded you can get a http code like 202. — Azarea, Jul 04 '18 at 07:26
@Azarea Thank you for clearing that up for me. How can I implement something that will give me HTTP 202 after processing each userid? — Roshan Upreti, Jul 04 '18 at 07:29
@RoshanUpreti if you want a http 202 after processing each userid, in this case you will have to make millions of http request, is this really worthwhile and practical? I'd prefer dump the upload link of each userid to db perhaps. — Azarea, Jul 04 '18 at 07:35
@Azarea you're absolutely right regarding the millions of requests. But, that is kind of my requirement. All rows processed need to return an HTTP 202. The background service definitely will consume a lot of I/O resources in the first run but after that, it will just snoop for the new incoming rows in the table. — Roshan Upreti, Jul 04 '18 at 07:46

score 0 · Accepted Answer · answered Jul 04 '18 at 07:52

0

If you want a http response after every row is processed, first you need to divide your method into which processes one row at one time, then you can use a Response to contain your http code and entity, like this:

@POST
@Path("attachments")
public Response moveToMinio() throws Exception {
    TiedostoDaoImpl tiedostoDao = new TiedostoDaoImpl();
    Integer userId = tiedostoDao.getOneCustomerId();
    String uploadLink = AttachmentService.insertAndUploadService(userId);

    return Response.status(Response.Status.ACCEPTED).entity(uploadLink).build();
}

answered Jul 04 '18 at 07:52

Azarea

456
5
19

Thank you so much for this. But wouldn't this only process one row at a time? I need to process the entire table. – Roshan Upreti Jul 04 '18 at 08:01
@RoshanUpreti But you can't have multi response in one http request – Azarea Jul 04 '18 at 08:06
does this mean that I need to send individual POST request for each userId? – Roshan Upreti Jul 04 '18 at 08:39
@RoshanUpreti yes, that's why i divide your method – Azarea Jul 04 '18 at 08:45
Thank you again. I have a small question though. One userId could have multiple rows in the table. How would it work in that case? – Roshan Upreti Jul 04 '18 at 09:02
@RoshanUpreti i think it depends on your requirement..either regard it as one userid or do sth else – Azarea Jul 04 '18 at 10:17

score 0 · Answer 2 · answered Jul 04 '18 at 13:18

0

Please refer to this How to run a background task in a servlet based web application?

Before you return response, put the job into a global queue and let the background process do the job.

answered Jul 04 '18 at 13:18

Dennis Kim

79
1
3

How can I send POST requests from a background service in Java?

2 Answers2