Removing Data From ElasticSearch

Question

I'm new to ElasticSearch. I'm trying to figure out how to remove data from ElasticSearch. I have deleted my indexes. However, that doesn't seem to actually remove the data itself. The other stuff I've seen points to the Delete by Query feature. However, I'm not even sure what to query on. I know my indexes. Essentially, I'd like to figure out how to do a

DELETE FROM [Index]

From PostMan in Chrome. However, I'm not having any luck. It seems like no matter what I do, the data hangs around. Thus far, I've successfully deleted the indexes by using the DELETE HTTP Verb in PostMan and using a url like:

   http://localhost:9200/[indexName]

However, that doesn't seem to actually remove the data (aka docs) themselves.

I check this with postman and got reposne as "{ "acknowledged": true }" If you see this acknowledged response don't worry. The index is removed from elastic. — bijayk, Sep 02 '16 at 12:42

score 491 · Answer 1 · edited May 12 '19 at 22:19

491

If you ever need to delete all the indexes, this may come in handy:

curl -X DELETE 'http://localhost:9200/_all'

Powershell:

Invoke-WebRequest -method DELETE http://localhost:9200/_all

edited May 12 '19 at 22:19

Jeroen Vannevel

41,258
21
92
157

answered Jun 20 '15 at 11:06

kha

16,333
9
28
57

24

this is very useful for development and needing to reset back to scratch (empty) database. Thanks!! – Artistan Sep 23 '15 at 18:55
3

in your bash_profile create an alias for this command and it will come in handy for development. – user805981 Jan 27 '16 at 18:15
2

'Wildcard expressions or all indices are not allowed' – ZurabWeb Apr 27 '16 at 22:03
1

Note that this will delete all data, including your x-pack access credentials. – Gajus Oct 16 '17 at 14:21
3

This is also deleting Kibana dashboards and visualizations – nano Oct 18 '17 at 11:26
I'm using AWS ES and when I run this command, I seem unable to recreate index patterns in Kibana UI. – James Webb Dec 01 '19 at 18:28

score 481 · Accepted Answer · answered Apr 08 '14 at 09:08

481

You can delete using cURL or visually using one of the many tools that open source enthusiasts have created for Elasticsearch.

Using cURL

curl -XDELETE localhost:9200/index/type/documentID

e.g.

curl -XDELETE localhost:9200/shop/product/1

You will then receive a reply as to whether this was successful or not. You can delete an entire index or types with an index also, you can delete a type by leaving out the document ID like so -

curl -XDELETE localhost:9200/shop/product

If you wish to delete an index -

curl -XDELETE localhost:9200/shop

If you wish to delete more than one index that follows a certain naming convention (note the *, a wildcard), -

curl -XDELETE localhost:9200/.mar*

Visually

There are various tools as mentioned above, I wont list them here but I will link you to one which enables you to get started straight away, located here. This tool is called KOPF, to connect to your host please click on the logo on top left hand corner and enter the URL of your cluster.

Once connected you will be able to administer your entire cluster, delete, optimise and tune your cluster.

answered Apr 08 '14 at 09:08

Nathan Smith

7,717
3
23
43

is there any way I can delete 3 doc's of which id I know. – HIRA THAKUR Jul 08 '15 at 06:18
@JayeshJain to my current knowledge, no. You could put 3 modified curl -XDELETE commands into a bash script and execute or run 3 one after the other. – Nathan Smith Jul 08 '15 at 10:39
@JayeshJain so curl -XDELETE localhost:9200/index/type/docid1 // curl -XDELETE localhost:9200/index/type/docid2 // curl -XDELETE localhost:9200/index/type/docid3 – Nathan Smith Jul 08 '15 at 10:40
i did it the same way.but I was just thinking if there is a smarter way of deleting multiple docs. I could use term if I knew the field. But In this scenario,i just need to delete docs by their id. Thx anyways – HIRA THAKUR Jul 08 '15 at 11:08
2

How can I delete an index with an invalid character, e.g., logstash-eu-%{customer}-2016.11.22. I want to delete ALL indices logstash-eu-%{customer}-* or logstash-eu-%* – Chris F Nov 29 '16 at 16:13
`curl: (7) Failed to connect to localhost port 9200: Connection refused` – Green Mar 06 '18 at 09:38
@Green your elasticsearch instance may not be listening on localhost? – Nathan Smith Mar 06 '18 at 10:51
Shouldn’t there be a space between the -X and DELETE ? – chmike Apr 28 '19 at 14:51

score 57 · Answer 3 · edited Jun 20 '20 at 09:12

The documentation (or The Definitive Guide) says, that you can also use the next query to delete all indices:

curl -XDELETE 'http://localhost:9200/*'

And there's an important note:

For some, the ability to delete all your data with a single command is a very scary prospect. If you want to eliminate the possibility of an accidental mass-deletion, you can set the following to true in your elasticsearch.yml:

action.destructive_requires_name: true

score 31 · Answer 4 · edited Jan 19 '17 at 08:42

31

You have to send a DELETE request to

http://[your_host]:9200/[your_index_name_here]

You can also delete a single document:

http://[your_host]:9200/[your_index_name_here]/[your_type_here]/[your_doc_id]

I suggest you to use elastichammer.

After deleting you can look up if the index still exists with the following URL: http://[your_host]:9200/_stats/

Good luck!

edited Jan 19 '17 at 08:42

Prasad Khode

5,897
11
38
52

answered Apr 08 '14 at 07:24

Jesper

529
4
9

what is the way to delete indices older than 10 days ? I can not use curator because my server is not support. – biolinh Sep 01 '17 at 09:19

score 25 · Answer 5 · answered Jun 22 '17 at 08:00

25

#list all index:       curl -XGET http://localhost:9200/_cat/indices?v

#delete index:         curl -XDELETE 'localhost:9200/index_name'
#delete all indices:   curl -XDELETE 'localhost:9200/_all'
#delete document   :   curl -XDELETE 'localhost:9200/index_name/type_name/document_id'

Install kibana. Kibana has a smarter dev tool which helps to build query easily.

answered Jun 22 '17 at 08:00

ysk

947
10
8

1

what is the way to delete indices older than 10 days ? I can not use curator because my server is not support. – biolinh Sep 01 '17 at 09:18

score 20 · Answer 6 · edited Sep 30 '16 at 23:48

20

Deleting the index will delete the mapping and type along. you can delete all rows by the following query

curl -XDELETE 'localhost:9200/twitter/tweet/_query?pretty' -d'
{
   "query": { 
      "match_all": 
   }
}'

However for above query you need to install delete-by-query plugin as of Elasticsearch's 2.0.0-beta1 delete-by-query was removed from main api

Install delete-by-query plugin

sudo bin/plugin install delete-by-query

For more

http://blog.appliedinformaticsinc.com/how-to-delete-elasticsearch-data-records-by-dsl-query/

edited Sep 30 '16 at 23:48

Bojan B

1,882
4
16
24

answered Sep 30 '16 at 18:44

shahid ashraf

201
2
4

Both before and after installing the plugin and restarting ES, I get "No handler found for uri and method". – Matthew Read Mar 04 '17 at 07:51
This does not work in Elasticsearch 6+. Use _delete_by_query instead. – Shailesh Pratapwar Feb 05 '19 at 08:19

score 13 · Answer 7 · edited Aug 02 '19 at 09:57

13

curl -X DELETE 'https://localhost:9200/_all'

Change http to https if you are using SSL certificate in you application

edited Aug 02 '19 at 09:57

RobC

16,905
14
51
62

answered Aug 02 '19 at 08:56

Vikash Chauhan

746
2
9
18

score 10 · Answer 8 · edited Apr 05 '17 at 21:25

10

You can delete an index in python as follows

from elasticsearch import Elasticsearch

es = Elasticsearch([{'host':'localhost', 'port':'9200'}])

es.index(index='grades',doc_type='ist_samester',id=1,body={
    "Name":"Programming Fundamentals",
    "Grade":"A"
})

es.indices.delete(index='grades')

edited Apr 05 '17 at 21:25

kgf3JfUtW

9,808
7
38
61

answered Apr 05 '17 at 11:51

Farid ullah

271
2
12

score 10 · Answer 9 · answered Aug 28 '17 at 15:11

simplest way !

Endpoint :
http://localhost:9201/twitter/_delete_by_query

Payload :
{
  "query": { 
    "match": {
      "message": "some message"
    }
  }
}

where twitter is the index in elastic search

ref ; https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html

Hubbitus · Answer 10 · 2018-08-25T23:49:14.883

8

For mass-delete by query you may use special delete by query API:

$ curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d '{
    "query" : {
        "term" : { "user" : "kimchy" }
    }
}

In history that API was deleted and then reintroduced again

Who interesting it has long history.

In first version of that answer I refer to documentation of elasticsearch version 1.6. In it that functionality was marked as deprecated but works good.
In elasticsearch version 2.0 it was moved to separate plugin. And even reasons why it became plugin explained.
And it again appeared in core API in version 5.0!

edited Aug 25 '18 at 23:49

answered Jul 31 '15 at 13:57

Hubbitus

4,481
2
37
42

3

Be careful using delete by query. Its deprecated for a major reason. OutOfMemoryError! – user3658423 Aug 08 '15 at 08:01
Sure. But you may spy if it happened for you or you have anough memory. – Hubbitus Aug 10 '15 at 09:52
1

This does not deprecated anymore: https://www.elastic.co/guide/en/elasticsearch/reference/6.4/docs-delete-by-query.html – Hubbitus Aug 25 '18 at 23:41

score 7 · Answer 11 · edited Aug 02 '19 at 08:50

7

To list down the indices curl -L localhost:9200/_cat/indices

9200 default port[change the port if using some other port]

You will likely find all indices starting with logstash-yyyy-mm-dd format(logstash-*)

You can see all the indices and use

To delete the indices and data trigger following command.

curl -XDELETE localhost:9200/index_name (Which will remove the data and indices both).

edited Aug 02 '19 at 08:50

sandip

435
6
19

answered May 21 '18 at 13:46

rajdeepbs29

721
8
8

score 6 · Answer 12 · edited Jul 14 '17 at 10:13

6

I wanted to delete logstash index and searched a lot regarding different tools like curl. But found the solution at the end. Login into Kibana. Go to Dev Tools tab and type DELETE /logstash-* in query field and hit green arrow button. if you get "acknowledged": true in response that means the data has been cleared.

edited Jul 14 '17 at 10:13

Manfred Radlwimmer

12,469
13
47
56

answered Jul 14 '17 at 09:44

Ashish Kumar

61
1
2

THANK YOU!!! i tried many other options- this is the only one that worked for me. – eladyanai Jan 30 '20 at 08:29

Gaurav · Answer 13 · 2019-10-17T09:46:55.807

6

You can delete either whole index,doc-type or a perticular id data. these are the three ways:

curl -XDELETE localhost:9200/index_name
curl -XDELETE localhost:9200/index_name/doc-type
curl -XDELETE localhost:9200/index_name/doc-type/documentId

and if you wish to delete all the index then go for wildcard.

edited Oct 17 '19 at 09:46

answered Sep 18 '17 at 11:00

Gaurav

409
6
14

Hi, i hope before executing queries you started your elastic search and make sure by default it binds to all local addresses. and instead of localhost you can also use your IP address. like 10.80.15.45:9200 And once check your ES setting I can think to check in network.bind_host and make sure it is either not set or is set to 0.0.0.0 or ::0 or to the correct IP address for your network. – Gaurav Mar 08 '18 at 06:33

score 6 · Answer 14 · answered Oct 15 '19 at 08:32

6

You can delete the index by Kibana Console:

To get all index:

GET /_cat/indices?v

To delete a specific index:

DELETE /INDEX_NAME_TO_DELETE

answered Oct 15 '19 at 08:32

FChiri

493
6
11

score 5 · Answer 15 · answered Nov 08 '16 at 12:55

5

There are lots of good answers here, but there is also something i'd like to add:

If you are running on AWS ElasticSearch service, you can´t drop/delete indexes. Instead of delete indexes, you must reindex them.

answered Nov 08 '16 at 12:55

Facundo La Rocca

3,105
1
21
41

Just deleted an index on AWS ElasticSearch, my domain is running ES 5.1. – gazarsgo Mar 31 '17 at 06:04
2

On AWS ES you can't open/close indexes - that requires reindexing. You can, however, delete indexes. I've only done it through the Kibana console, but it definitely works. – Tom Dufall Aug 30 '19 at 10:39

score 4 · Answer 16 · answered Mar 30 '18 at 09:42

You can also delete the index using DELETE action in 'elasticsearch head' (Chrome plugin ). Add it to your chrome and connect it to your host. There you will find all your indices and if you click on actions button below the index you want to delete, you will find a DELETE option in the drop down. click on it and enter DELETE in the pop-up. Your index will be deleted. 'Elasticsearch head' extension is an easy way to view and manage your indices and data.

score 3 · Answer 17 · answered Mar 05 '20 at 12:47

You can delete one or more indices, which really deletes their files from disk. For example:

curl -XDELETE localhost:9200/$INDEXNAME

Where $INDEXNAME can be an index name (e.g. users_v2), N indices separated by comma (e.g. users_v2,users_v3). An index pattern (e.g. users_*) or _all, also works, unless it's blocked in the config via action.destructive_requires_name: true.

Deleting individual documents is possible, but this won't immediately purge them. A delete is only a soft delete, and documents are really removed during segment merges. You'll find lots of details about segments and merges in this presentation. It's about Solr, but merges are from Lucene, so you have the same options in Elasticsearch.

Back to the API, you can either delete individual documents by ID (provide a routing value if you index with routing):

curl -XDELETE localhost:9200/users_v2/_doc/user1

Or by query:

curl -XPOST -H 'Content-Type: application/json' localhost:9200/users_v2/_delete_by_query -d '{
  "query": {
    "match": {
      "description_field": "bad user"
    }
  }
}'

funbrain9 · Answer 18 · 2021-02-03T01:34:57.237

1

You can also use chrome extension elasticsearch-head to delete index

edited Feb 03 '21 at 01:34

answered Sep 02 '19 at 03:27

funbrain9

367
3
13

score 0 · Answer 19 · answered May 03 '20 at 08:19

Say I need to delete an index filebeat-7.6.2-2020.04.30-000001 and I performed it using a curl DELETE option (curl -X DELETE "localhost:9200/filebeat-7.6.2-2020.04.30-000001?pretty") and results in an authentication problem as below;

{
  "error" : {
    "type" : "security_exception",
    "reason" : "missing authentication credentials for REST request [/filebeat-7.6.2-2020.04.30-000001?pretty]"
  },
  "status" : 401
}

Here you should authenticate the curl request using the username and password you have provided for Elasticsearch. Try then

curl -X DELETE -u myelasticuser:myelasticpassword "localhost:9200/filebeat-7.6.2-2020.04.30-000001?pretty"

will results in { "acknowledged" : true }.

score 0 · Answer 20 · answered May 05 '21 at 06:04

0

Delete by query API

Deletes documents that match the specified query.

Example:

POST /my-index-000001/_delete_by_query
{
  "query": {
      "match": {
           "user.id": "elkbee"
      }
   }
}

Reference : ES Guide >> Delete by query API

answered May 05 '21 at 06:04

Pramod H G

916
8
12

Removing Data From ElasticSearch

20 Answers20

In history that API was deleted and then reintroduced again

Linked