Submit Feedback on Extractions

Feedback is approved or corrected extraction values that are submitted by you back to the extraction system.

You should always submit the feedback on extractions. The feedback that is correctly submitted lets Gini measure the extraction quality in order to provide the best possible user experience. Moreover, it helps us continuously improve the extraction system and perform valuable data analytics for you.

The feedback should only be sent if real users see, approve, correct, and complement the extraction-based data.

Gini employs various machine learning techniques in order to learn from feedback automatically. Therefore, it is equally important for Gini to receive both feedback on correct and incorrect extractions.

Ways of submitting feedback

There are currently two ways to submit the feedback.

The first and the most common one is to submit the complete feedback in one request. This is the easiest way if your frontend (application) displays the extractions on a screen in an editable form. A user can modify the extractions before pressing the confirmation button.

The second (rarely used) way is where the final approvement signal (button click) is not possible. In such case, send the feedback on one label per request.

Types of feedback

There are three different types of feedback:

positive feedback (preferred): the extraction was correct and confirmed by the user without modification.
complementary feedback: the extraction system extracted nothing, for example, the response doesn't contain the requested label. The user manually entered the correct value.
negative feedback: the extraction was incomplete or erroneous. The user manually corrected the extraction.
informative feedback: the extraction was changed for another reason, for example, if a line item is returned, the quantity extraction gets feedback "0" and a return reason.

The Gini Pay API lets you submit feedback on multiple extractions for a single document with a single request. It is strongly recommended that you submit your feedback in this way for two reasons. On the one hand, the total number of round trips is reduced to one and the feedback is handled internally as a batch. Therefore, the update is more efficient for multiple extractions compared to submitting the feedback with each separate request. On the other hand, Gini's machine learning training techniques can benefit from the feedback on multiple extractions since Gini is aware of the fact that single parts of the submitted feedback belong together.

Send feedback only on those labels that are physically seen by the user. Unseen labels should be filtered out.

//submitting feedback on extractions


    PUT /documents/{id}/extractions/feedback

Request

Headers

Header	Value
`Content-Type`	`application/vnd.gini.v1+json`

Body

Key	Type	Description
`extractions`	`object`	Feedback on atomic extractions.
`compoundExtractions`	`object`	Feedback on compound extractions.

Response

Status Code	Description
204 (No Content)	The feedback was successfully processed.
404 (Not Found)	The document or the label could not be found.
406 (Not Acceptable)	The request headers were not set correctly.
422 (Unprocessable Entity)	At least one value was not valid regarding entity validation rules of the label.

Test example

In order to test the integration of the feedback functionality, we offer a PDF test document with the expected outcome in a JSON format. This document can be used for an integration test of Gini’s Pay API in general. Furthermore, we provide a query for sending feedback where all fields are correct except for the amountToPay. After sending this feedback, we show you how to ensure that the feedback with your confirmation and correction for the field values was successful.

Sending the test document to the Gini Pay API. See Submit Files.
Requesting the results. See Retrieve Extractions.
You should get a JSON like in the example.
Correcting the document with the following JSON. Here we change the amountToPay from "995.00:EUR" to "950.00:EUR" and confirm the other field values (iban, bic, paymentRecipient, paymentReference).

{
   "feedback": {
       "amountToPay": {
           "value": "950.00:EUR"
       },
       "iban": {
           "value": "DE68130300000017850360"
       },
       "bic": {
           "value": "BYLADEM1001"
       },
       "paymentPurpose": {
           "value": "RE-20170512-02"
       },
       "paymentRecipient": {
           "value": "Fahrrad Rückenwind"
       }
   }
}

//Example

//We show a more elaborated example here in order to explain different types of the feedback. The example scenario is as follows: the user uploads a document where the labels amountToPay, paymentReference, iban are extracted. Unfortunately the label paymentRecipient could not be extracted. The response to the extractions request is as follows:

{
    "candidates": {
    },
    "extractions": {
        "amountToPay": {
            "box": {
                "height": 8.0,
                "left": 545.0,
                "page": 1,
                "top": 586.0,
                "width": 17.0
            },
            "candidates": "amounts",
            "entity": "amount",
            "value": "5.60:EUR"
        },
        "iban": {
            "box": {
                "height": 7.0,
                "left": 447.0,
                "page": 1,
                "top": 746.0,
                "width": 100.0
            },
            "candidates": "ibans",
            "entity": "iban",
            "value": "DE68130300000017850360"
        },
        "paymentReference": {
            "entity": "reference",
            "value": "ReNr 123, KdNr 32"
        }
    },
    "compoundExtractions": {
        "lineItems": [
            {
              "artNumber": {
                "value": "10101",
                "entity": "text" ,
                "box": {
                    "height": 7.0,
                    "left": 55.0,
                    "page": 1,
                    "top": 546.0,
                    "width": 100.0
                }
              },
              "quantity": {
                "value": "12",
                "entity": "numeric"
              }
            },
            {
              "artNumber": {
                "value": "10103",
                "entity": "text"
              },
              "quantity": {
                "value": "3",
                "entity": "numeric"
              }
            }
        ]
    }
}

//The user adds missing paymentRecipient value (complementary feedback) and corrects the paymentReference to "ReNr 1735, KdNr 37" (negative feedback). Sets the quantity of one line item to 0 and adds a reason for returning the articles. The iban, amountToPay and the remaining part of lineItems are correct (positive feedback). The document is not shown, so we can leave out the boxes. Then the resulting feedback request is as follows:

{
   "extractions": {
       "amountToPay": {
           "value": "5.60:EUR"
       },
       "iban": {
           "value": "DE68130300000017850360"
       },
       "paymentReference": {
           "value": "ReNr 1735, KdNr 37"
       },
       "paymentRecipient": {
           "value": "Zalando SE"
       }
   },
 "compoundExtractions": {
        "lineItems": [
            {
              "artNumber": {
                "value": "10101"
              },
              "quantity": {
                "value": "0"
              },
              "returnReason": {
                "value": "r3"
              }
            },
            {
              "artNumber": {
                "value": "10103"
              },
              "quantity": {
                "value": "3"
              }
            }
        ]
    }
}
Give feedback and correct or verify multiple specific labeled extraction patterns with a single PUT or POST request to the document extractions URI.

The labels must correspond to the names of the extraction types e.g. amountToPay. See available specific extractions for possible values.

Headers
Header	Value
Content-Type	application/vnd.gini.v1+json
Body
Key	Type	Description
extractions	object	Feedback on atomic extractions
compoundExtractions	object	Feedback on compound extractions
 The boxes are optional. Boxes should only be sent if your use case visualize the document with the locations of the extractions. Thus, the user can confirm the correct location of a extraction.
Response
Status Code	Description
204 (No Content)	The feedback was successfully processed.
404 (Not Found)	The document or the label could not be found.
406 (Not Acceptable)	The request headers were not set correctly.
422 (Unprocessable Entity)	At least one value was not valid regarding entity validation rules of the label.
Test Example
In order to test the integration of the feedback functionality, we offer a PDF test document with the expected outcome in a JSON format. This document can be used for an integration test of Gini’s Pay API in general. Furthermore, we provide a query for sending feedback where all fields are correct except for the amountToPay. After sending this feedback we show you how you can ensure that the feedback with your confirmation and correction for the field values was successful.

Sending the test document to the Gini Pay API (see also submitting files)
Requesting the results (see also retrieving extractions)
You should get a JSON like in example.
Correcting the document with the following JSON (see also submitting feedback on extractions). Here we change the amountToPay from "995.00:EUR" to "950.00:EUR" and confirm the other field values (iban, bic, paymentRecipient, paymentReference).
{
   "feedback": {
       "amountToPay": {
           "value": "950.00:EUR"
       },
       "iban": {
           "value": "DE68130300000017850360"
       },
       "bic": {
           "value": "BYLADEM1001"
       },
       "paymentPurpose": {
           "value": "RE-20170512-02"
       },
       "paymentRecipient": {
           "value": "Fahrrad Rückenwind"
       }
   }
}

Submit feedback on invalid extractions

Request

Response

Status Code	Description
204 (No Content)	Label removal was successful.
404 (Not Found)	The document or the label could not be found.

//submitting feedback on invalid extractions

    DELETE /documents/{id}/extractions/{label}