Like HTML, it is designed around common web standards and technologies, with additional provisions for describing the shape, location, and content of the world at large. Just as a webpage describes content meant for display in a browser window, RML maps content (photos, panoramas, geometry, and more) directly to the world, in 2D and 3D, for display and/or augmentation.

Like Javascript, Reality Markup Language is designed to be flexible and resilient, largely free of hard-coded schemas, specs, and centralized definitions that tend to restrict creativity and innovation. The goal is simplicity and ease of use: expose the inter-collection of many terabytes of information in a common easy-to-traverse metadata format. Think of these as the slightly more elaborate and descriptive weblinks and metadata for the real world.

RML services serve and/or point to this data in a variety of forms, from simple REST calls and CDN stores to streaming HTTP updates designed for real-time rendering of the world and augmented reality. Conceptually, one can think of there being a dynamic and vastly-interconnected web of geospatial data, servers that serve it up as RML and connect it better, and then clients everywhere that sample small roaming windows of this giant web for local rendering, exploration, or still as yet unimagined applications.

Our core aim is not to re-invent the web or core web standards, but to add enough spatial know-how to make web concepts map to the world in all its natural richness. Our goal is to provide a base and set of primitive constructs on which the world can not only hang its data (in a way that is neither “walled-garden” nor owned by any single entity, including Microsoft) but also build mashups, with innovative capabilities that both read and augment the world graph.

An RML-capable web-app should look and behave largely like a typical web-app, with the addition of spatial content and the ability to transform or interpret that content (and render in 2D or 3D) based on a user’s actual or virtual location in the world. We typically serve the data in JSON form for Javascript consumption, along with optional JS open sourced libraries that can more robustly pack and unpack this data using a simple syntax. This language allows us to write RML data today that we can be fairly sure will be readable independent of how the format and uses change over time.

A typical RML JSON definition looks like this:

  “type”: “Entity”,
  “id” : “12342342”,
  “transform”: { “worldCoord”: { “lat”: -122.3121, “lon”: 47.234 } },
  “info”: { “author”: “avi”, “license”: 3 },
  “sources”: [
      “type” : “Image”,
      “url”: "”

Here, we’re taking extra care to be descriptive so that future parsers can re-interpret these fields as needed. We need only specify a common definition for the fact that ‘info’ contains the textual metadata about this item, such as author, copyright, creation date and so on. If tomorrow, people add a textual description to ‘info’, one simply adds that field and existing code is no worse for it. This is the power of NoSQL databases and schema-less formats.

Here, “transform” objects are designed to accommodate a variety of coordinate systems, from the common Mercator Lat/Long to local “child” 3D/6DOF position and orientation, even scale. Hierarchy primitives are also designed into the system to accommodate local or chained coordinate systems, constraint systems, and so on.

Sources can be anything, from images to video, panoramas, whole virtual environments, and so on. We do not try to model or redefine all of the formats required for specifying these complex objects, but merely link to them as they exist today. As such, you could see a RML entity point to a KML file and reference a named object within. You could see RML contain the list of raw sources for some mashup or composition. RML can be easily created from Open Graph markup to help push a restaurant’s geo-aware web page into the common world graph.

The API for accessing RML is also meant to be simple. The way one gets an RML entity is as follows:

[GET] -> returns stored object

(note: none of these URLs are publicly accessible entry points yet. APIs are subject to change – please check back for updates or subscribe to our RSS feed)

This should return the entity with the given ID. Optional parameters, like “?traverse” are added to control how much neighboring or connected information one wants the server to return with each call. The default is just the entity itself. But we also generate image matches and other “annealing” information that grows the database over time. Asking for some degree of traversal can obtain a set of entities such that a client need not need to ask again for those associated bits.

Updating an existing RML entity is as simple as:

[POST] -> returns updated object

Here, the body or “data” section of this HTTP message contains the new or partial (to be merged, field by field) RML information in JSON format (for now, XML at a later date).

Note: IDs are strings, but are merely required to be globally unique. Uploading an RML entity with a dummy ID or local name references is supported (these will be rewritten with GUIDs by the server), but it’s generally better to obtain IDs in advance of a POST. The RML service will give you one or more globally unique IDs by calling:

[POST] [&count=(#number)] -> returns JSON ID or list of IDs

All “write” APIs currently require an API key, specified as “&key=xxx” on the parameter line. RML may in the future offer scoped visibility of entities, requiring some form of authentication to reach certain user-protected items. However, for now, we consider all of RML to be publicly accessible for read-only purposes. Further details of the “write” part of the system will be announced at a future date.

One thing people often want to do is search for entities, and we try to make that easy.

[GET] [&x1=(ang) &y1=(ang) &x2=(ang)&y2=(ang)] [&t1=(time)] [&t2=(time)]


[GET] [&lon_min=(ang) &lat_min=(ang) &lon_max=(ang) &lat_max=(ang)] [&sec_min=(time)] [&sec_max=(time)]

Here, we specify a bounding box in degrees (+/-180, +/-90), where X represents the longitude and Y the latitude min and max. T1 and T2 represent a min/max time window as well, currently in double-precision server seconds, for which RML will filter out results live to only within that time window. The result here and for any list-like results is a JSON object containing a JSON array called ‘elements’ plus some additional tracking information, such as the query time, context, and timestamp of the call.

These calls are intended to be fast. However, much of RML is aggregated from remote or federated sources and some of these take a long time to integrate. Often, such a call can kick off or update a long-running process to obtain more information for a given area.

In RESTful mode, we could wait until all of that information is obtained for a single complete call. In Async mode, we return immediately with the information RML has already cached. A later query or persistent stream connection can supply the rest. In the case of a future RESTful call, one can simply ask for elements newer than the first call, which will return zero until the background integration completes. But in streaming mode, we leave the connection open, flushing early results for consumption on the client but waiting for all results or additional operations to complete.

And we support other searches, such as :


Here, we aim to expose as much of the internal query language, as we can, either in URL or parameter form, without sacrificing performance , scalability, or security, especially of users’ own data.

For example, one great feature we hope to deploy soon is place names. Given a URL like:


We can return all of the RML for New York as of some regular package interval. The file will reflect that package time so a client can easily and nominally query for anything new since that time. These are designed to be internet cache-friendly, such that our servers need to do very little work for commonly read data. However, freshness is the tradeoff here.


In RML, everything is described as either Entities and Relations. Entities are the nouns and adjectives that describe places, things, and other artifacts. Relations are the verbs and adverbs, connecting Entities in interesting ways.

For example, Transforms are a kind of Relation that we tend to store with the entity for simplicity, however, we allow for multiple independent Transforms to point to the same Entity, forming an instancing model, where the same conceptual Entity can appear in multiple photos, or even multiple places in the world.

Other Relations are more like annotated sets. Match Relations describe which Entities are similar (in terms of image matching) to which and by how much. We obtain these just like Entities, as they have their own unique ID. Spatial searching will not find relationships alone, but traversing from entities will. In general, the RML server is designed to make sure that a package sent to a client contains all forward references to Entities and Relationships fully described, such that only one call is needed.


We will encourage user-contribution of new fields to store in RML objects. The only requirement we put is that RML objects support a data contract identifier that allows a client or server to fully traverse the object and potentially translate from one variation to another. We’ll specify the basic format for this and provide minimal enforcement where folks go astray. But in general, we want to see RML objects become as rich as varied as the data they point to.


We also encourage community involvement from day one. If you’re interested in learning more about RML, we’ll shortly be setting up the usual allotment of social media to communicate effectively with all of you.  Look for us on Facebook and Twitter for starters! And of course check back here.