For object and entity export format details, refer to these sections:
- Standard object detection
- Object detection in video assets
- Pose estimation
- Named Entity Recognition
- Named Entity Recognition in PDFs
For general export format information, refer to Exported data format.
Standard object detection
This structure applies to geospatial imagery as well. The only difference is that in geospatial labels
xstands for longitude andyfor latitude.
Check the detailed JSON property descriptions for 2D labeling tools
annotationsList of annotationsboundingPoly: Polygon of the object contournormalizedVertices: List of normalized (relative to the original image: range from 0 to 1) vertices of the polygon. In geospatial labels `x` stands for longitude and `y` stands for latitude.x: Abscissa of the vertex position from top lefty: Ordinate of the vertex position from top leftvertices: List of vertices of the polygon, listed as pixel coordinates (built by multiplying the normalized coordinates by the asset resolution). In geospatial labels `x` stands for longitude and `y` stands for latitude.x: Abscissa of the vertex position from top lefty: Ordinate of the vertex position from top leftcategories: Category of the objectname: Name of the categoryconfidence: Confidence (100 by default when done by human)mid: A unique identifier of the objectscore: When a pre-annotation model is used, the score is the confidence of the object detectiontype: Type of tool used for the annotation (one of: "rectangle", "polygon", or "semantic")
Examples of exported standard 2D object detection jobs:
"json_response": {
"JOB_0": {
"annotations": [{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"categories": [{ "name": "TESLA", "confidence": 100 }],
"mid": "unique-tesla",
"type": "rectangle",
}]
}
}
'json_response': {
"JOB": {
"annotations": [{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"categories": [{ "name": "FERRARI", "confidence": 100 }],
"mid": "unique-ferrari",
"type": "rectangle",
"children": {
"CLASSIFICATION_JOB": {
"categories": [{ "name": "GREY", "confidence": 100 }]
}
}
}]
}
}
'json_response': {
"JOB_0": {
"annotations": [{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"mid": "car",
"type": "rectangle",
"categories": [{ "name": "WHOLE_CAR", "confidence": 100 }],
},
{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"mid": "left-front-wheel",
"type": "rectangle",
"categories": [{ "name": "WHEELS", "confidence": 100 }],
},
{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"mid": "left-back-wheel",
"type": "rectangle",
"categories": [{ "name": "WHEELS", "confidence": 100 }],
}]
},
'RELATION_JOB': {
'annotations': [
{
'categories': [{'name': 'WHOLE_CAR_AND_WHEELS', 'confidence': 100}],
'startObjects': [{'mid': 'car'}],
'endObjects': [{'mid': 'left-front-wheel'}, {'mid': 'left-back-wheel'}],
},
]
},
}
'json_response': {
"JOB_0": {
"annotations": [{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"mid": "unique-tesla",
"type": "polygon",
"categories": [{ "name": "TESLA", "confidence": 100 }],
}]
}
}
'json_response': {
"JOB_0": {
"annotations": [{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"mid": "unique-tesla",
"type": "semantic",
"categories": [{ "name": "TESLA", "confidence": 100 }],
}]
}
}
If the image has been rotated, the
json_responsecontains an additional top-levelROTATION_JOBwith information on the angle that the image was rotated by. For example:'ROTATION_JOB': {'rotation': 90}. After rotation, all the coordinates of a bounding box get automatically adjusted.
Check the detailed JSON property descriptions for 1D labeling tools
annotationsList of annotationscategories: Category of the objectconfidence: Confidence (100 by default when done by human)name: Name of the categorymid: A unique identifier of the objectchildren: Nested jobspointorpolyline: Normalized (relative to the original image: range from 0 to 1) coordinates of the point or line. In geospatial labels `x` stands for longitude and `y` stands for latitude.x: Point abscissay: Point ordinatepointPixelsorpolylinePixels: Coordinates of the point, line, or vector, listed as pixel coordinates (built by multiplying the normalized coordinates by the asset resolution) In geospatial labels `x` stands for longitude and `y` stands for latitude.x: Point abscissay: Point ordinatetype: Type of tool used for the annotation (one of: "marker", "vector", or "polyline")
Examples of exported standard 1D object detection jobs:
"jsonResponse": {
"OBJECT_DETECTION_JOB_0": {
"annotations": [
{
"categories": [
{
"confidence": 100,
"name": "POINT"
}
],
"children": {},
"mid": "20220701113558339-67037",
"point": {
"x": 0.6396378516199591,
"y": 0.2192328091653808
},
"pointPixels": {
"x": 0.8727983223923027 * 474,
"y": 0.2035857007889187 * 842,
},
"type": "marker"
}
]
}
}
"jsonResponse": {
"OBJECT_DETECTION_JOB_2": {
"annotations": [
{
"categories": [
{
"confidence": 100,
"name": "LINE"
}
],
"children": {},
"mid": "20220701115017888-45281",
"polyline": [
{
"x": 0.38086698287342297,
"y": 0.44925135916230174
},
{
"x": 0.4255140693825084,
"y": 0.6355339876809349
}
],
"polylinePixels": [
{
"x": 0.6555950869111132 * 474,
"y": 0.2574428654046088 * 842
},
{
"x": 0.5659240263913562 * 474,
"y": 0.17890116700672754 * 842
},
],
"type": "polyline"
}
]
}
}
"jsonResponse": {
"OBJECT_DETECTION_JOB": {
"annotations": [
{
"categories": [
{
"confidence": 100,
"name": "LINE"
}
],
"children": {},
"mid": "20220701113556786-8668",
"polyline": [
{
"x": 0.28428348960887073,
"y": 0.33424208416384127
},
{
"x": 0.4264252344133061,
"y": 0.2548694859254671
}
],
"polylinePixels": [
{
"x": 0.6555950869111132 * 474,
"y": 0.2574428654046088 * 842
},
{
"x": 0.5659240263913562 * 474,
"y": 0.17890116700672754 * 842
},
],
"type": "vector"
}
]
}
}
Object detection in video assets
- In object detection jobs, native video and frame assets use an additional
isKeyFrameproperty. This is a Boolean indicating if the timestamp or frame is used for interpolation.- If you are importing labels into a Video project, the
midfield (id of an object detection annotation) cannot be longer than 50 characters.
Examples of exported object detection video jobs:
"jsonResponse": {
"0": { //Timestamp number
"job_0": {
"annotations": [
{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"categories": [{ "confidence": 100, "name": "BIG" }],
"isKeyFrame": true,
"mid": "2020110316040863-98540",
"score": null,
"type": "rectangle"
}
}
}
}
"jsonResponse": {
"0": { //Frame number
"job_0": {
"annotations": [
{
"boundingPoly": [
{
"normalizedVertices": [
{
"x": 0.7412807669633257,
"y": 0.11831185681407619
},
{
"x": 0.7412807669633257,
"y": 0.07455291056382807
},
],
"vertices": [
{
"x": 0.7412807669633257 * 474,
"y": 0.11831185681407619 * 842,
},
{
"x": 0.7412807669633257 * 474,
"y": 0.07455291056382807 * 842,
},
],
}
],
"categories": [{ "confidence": 100, "name": "BIG" }],
"isKeyFrame": true,
"mid": "2020110316040863-98540",
"score": null,
"type": "rectangle"
}
}
}
}
Pose Estimation
Check the detailed JSON property descriptions
annotationsList of annotationscategories: Category of the objectname: Name of the categorychildren: Nested jobsmid: A unique identifier of the objectpoints: List of the points composing the objectcode: Identifier (unique for each point in an object)mid: Id of the pointpoint: Normalized (relative to the original image: range from 0 to 1) coordinates of the pointx: Point abscissay: Point ordinatetype: Tool used to annotate the point. In pose estimation jobs, it's `marker`type: Tool used to annotate the pose. In pose estimation jobs, it's `pose`
Examples of exported post estimation jobs:
"jsonResponse": {
"JOB_0": {
"annotations": [
{
"categories": [
{
"name": "HEAD"
}
],
"children": {},
"mid": "20220511111820SS-508",
"points": [
{
"children": {},
"code": "NOSE",
"mid": "20230220105343039-43309",
"point": {
"x": 0.16032453806928879,
"y": 0.43348072704538043
},
"type": "marker"
},
{
"children": {},
"code": "LEFT_EYE",
"mid": "20230220105343039-34486",
"point": {
"x": 0.21863386451307548,
"y": 0.3704089988436732
},
"type": "marker"
},
{
"children": {},
"code": "LEFT_EARBASE",
"mid": "20230220105343039-78183",
"point": {
"x": 0.2556378601408632,
"y": 0.32873660699611673
},
"type": "marker"
}
],
"type": "pose"
}
]
}
}
"annotations": [
{
"categories": [
{
"name": "HEAD"
}
],
"children": {},
"mid": "20230220115140640-7038",
"points": [
{
"children": {
"CLASSIFICATION_JOB": {
"categories": [
{
"confidence": 100,
"name": "CATEGORY_1"
}
]
}
},
"code": "NOSE",
"mid": "20230220105343039-43309",
"point": {
"x": 0.16032453806928879,
"y": 0.43348072704538043
},
"type": "marker"
},
{
"children": {},
"code": "LEFT_EYE",
"mid": "20230220105343039-34486",
"point": {
"x": 0.21863386451307548,
"y": 0.3704089988436732
},
"type": "marker"
},
{
"children": {},
"code": "LEFT_EARBASE",
"mid": "20230220105343039-78183",
"point": {
"x": 0.2556378601408632,
"y": 0.32873660699611673
},
"type": "marker"
}
],
"type": "pose"
},
Named entity recognition (NER)
Check the detailed JSON property descriptions
annotationsList of annotationsbeginOffset: Position of the entity mentioncategories: Category of the objectname: Name of the categoryconfidence: Confidence (100 by default when done by human)content: Content of the mentionmid: A unique identifier of the object
Examples of exported named entity recognition jobs:
"json_response": {
"JOB": {
"annotations": [
{"categories": [{ "name": "NAME", "confidence": 100 }],
"beginOffset": 0,
"content": "Chicago Bulls",
"mid": "chicago"},
{"categories": [{ "name": "NAME", "confidence": 100 }],
"beginOffset": 30,
"content": "Jerry Krause",
"mid": "krause"},
{"categories": [{ "name": "NAME", "confidence": 100 }],
"beginOffset": 63,
"content": "Gatorade",
"mid": "gatorade"},
{"categories": [{ "name": "NOUN", "confidence": 100 }],
"beginOffset": 22,
"content": "Manager",
"mid": "manager"},
{"categories": [{ "name": "NOUN", "confidence": 100 }],
"beginOffset": 14,
"content": "General",
"mid": "general"},
{"categories": [{ "name": "NOUN", "confidence": 100 }],
"beginOffset": 84,
"content": "medicine",
"mid": "medicine"},
{"categories": [{ "name": "NOUN", "confidence": 100 }],
"beginOffset": 104,
"content": "players",
"mid": "players"},
{"categories": [{ "name": "NOUN", "confidence": 100 }],
"beginOffset": 116,
"content": "coaches",
"mid": "coaches"},
{"categories": [{ "name": "VERB", "confidence": 100 }],
"beginOffset": 124,
"content": "milled",
"mid": "milled"},
{"categories": [{ "name": "VERB", "confidence": 100 }],
"beginOffset": 43,
"content": "was standing",
"mid": "standing"},
{"categories": [{ "name": "NOUN", "confidence": 100 }],
"beginOffset": 96,
"content": "hand",
"mid": "hand"},
{"categories": [{ "name": "NOUN", "confidence": 100 }],
"beginOffset": 72,
"content": "cooler",
"mid": "cooler"}
]
}
}
'json_response': {
"JOB": {
"annotations": [
{
"beginId": 'PART1',
"beginOffset": 14,
"categories": [{ "name": "NOMINAL_GROUP", "confidence": 100 }],
"content": "Declaration of the thirteen United States of America",
"endId": 'PART2',
"endOffset": 41,
"mid": "declaration",
},
]
}
}
'json_response': {
"NAMED_ENTITIES_RECOGNITION_JOB": {
"annotations": [
{"categories": [{ "name": "SUBJECT", "confidence": 100 }],
"beginOffset": 0,
"content": "Jordan",
"mid": "Jordan"},
{"categories": [{ "name": "VERB", "confidence": 100 }],
"beginOffset": 84,
"content": "was wearing",
"mid": "wearing"},
{"categories": [{ "name": "VERB", "confidence": 100 }],
"beginOffset": 111,
"content": "tucked",
"mid": "tucked verb"},
{"categories": [{ "name": "COMPLEMENT", "confidence": 100 }],
"beginOffset": 96,
"content": "a blue sweater tucked into high-rise pants",
"mid": "blue sweater complement"},
{"categories": [{ "name": "VERB", "confidence": 100 }],
"beginOffset": 47,
"content": "peered down",
"mid": "peered"},
{"categories": [{ "name": "COMPLEMENT", "confidence": 100 }],
"beginOffset": 62,
"content": "the hefty Krause",
"mid": "Krause complement"},
{"categories": [{ "name": "SUBJECT", "confidence": 100 }],
"beginOffset": 62,
"content": "the hefty Krause",
"mid": "Krause subject"}
]
},
"NAMED_ENTITIES_RELATION_JOB": {
"annotations": [
{"categories": [{ "name": "VERB_AND_SUBJECT_S", "confidence": 100 }],
"startEntities": [{ "mid": "peered" }],
"endEntities": [{ "mid": "Jordan" }]},
{"categories": [{ "name": "VERB_AND_COMPLEMENT_S", "confidence": 100 }],
"startEntities": [{ "mid": "peered" }],
"endEntities": [{ "mid": "Krause complement" }]},
{"categories": [{ "name": "VERB_AND_SUBJECT_S", "confidence": 100 }],
"startEntities": [{ "mid": "wearing" }],
"endEntities": [{ "mid": "Krause subject" }]},
{"categories": [{ "name": "VERB_AND_COMPLEMENT_S", "confidence": 100 }],
"startEntities": [{ "mid": "wearing" }],
"endEntities": [{ "mid": "blue sweater complement" }]}
]
}
}
NER in PDFs
Annotation structure for NER in PDFs is different. Instead of
beginOffset, the annotations work with the coordinates of the polygon that the data belongs to and the page number.
Check the detailed JSON property descriptions
annotations: List of annotationsannotations: List of positions of the annotation (for NER, when an annotation spans multiple lines, there will be multiplepolysand a singleboundingPoly)boundingPoly: Object contournormalizedVertices: List of vertices of the object contour. Coordinates normalized (relative to the original image: range from 0 to 1, expressed as a percentage of the total document size).x: Abscissa of the vertex position from top lefty: Ordinate of the vertex position from top leftvertices: List of vertices of the object contour. Coordinates listed as pixel coordinates (built by multiplying the normalized coordinates by the asset resolution.x: Abscissa of the vertex position from top lefty: Ordinate of the vertex position from top leftpageNumberArray: The numbers of pages where the annotation appearspolys: Coordinates of the different rectangles in the annotation. An annotation can have several rectangles (for example if the annotation covers more than one line)categories: Category of the objectname: Name of the categoryconfidence: Confidence (100 by default when done by human)mid: A unique identifier of the object
Example of an exported Named Object Recognition job with PDF:
"json_response": {
"NAMED_ENTITIES_RECOGNITION_JOB": {
"annotations": [{
"annotations": [{
"boundingPoly": [
{
"normalizedVertices": [
[
{
"x": 0.24565469293163383,
"y": 0.07735917978757553
},
{
"x": 0.24565469293163383,
"y": 0.14694151081343712
},
{
"x": 0.40961761297798377,
"y": 0.14694151081343712
},
{
"x": 0.40961761297798377,
"y": 0.07735917978757553
}
]
],
"vertices": [
[
{
"x": 367.74507531865584,
"y": 163.92410196987257
},
{
"x": 367.74507531865584,
"y": 311.36906141367325
},
{
"x": 613.1975666280417,
"y": 311.36906141367325
},
{
"x": 613.1975666280417,
"y": 163.92410196987257
}
]
]
}
],
"polys": [
{
"normalizedVertices": [
[
{
"x": 0.24565469293163383,
"y": 0.07735917978757553
},
{
"x": 0.24565469293163383,
"y": 0.14694151081343712
},
{
"x": 0.40961761297798377,
"y": 0.14694151081343712
},
{
"x": 0.40961761297798377,
"y": 0.07735917978757553
}
]
],
"vertices": [
[
{
"x": 367.74507531865584,
"y": 163.92410196987257
},
{
"x": 367.74507531865584,
"y": 311.36906141367325
},
{
"x": 613.1975666280417,
"y": 311.36906141367325
},
{
"x": 613.1975666280417,
"y": 163.92410196987257
}
]
]
}
],
"pageNumberArray": [
1
]
}],
"categories": [{"name": "TITLE", "confidence": 100}],
"content": "Learning Active Learning from Data",
"mid": "article-title"
}],
}
}