Lytics REST API Documentation

Learn more about our efficient, adaptable, and scalable REST API.

Lytics API

The Lytics API is a restful JSON api that includes:

  • Data Collection api’s for collection, and upload of custom data.

  • Personalization api for real-time user profile usage in personalization.

  • Segmentation api for lists of users, and creating/managing the segmentation rules.

  • Catalog api for schema information.

  • Content api for content recommendation, and content-classification to drive personalization.

  • Management api for general account management.

Authentication

The Lytics API supports authentication using one of the following:

Login to your account Lytics App and navigate to Account to find your keys. After you have acquired your token, use it to access the Lytics API.

Our api supports two methods for authorization:

  • query string url parameter, using access_token

  • http Authorization HEADER

# example showing passing auth token in header
curl -XPOST 'https://api.lytics.io/api/segment' \
    -H "Authorization: pretendtoken8762" \
    -H 'Content-type: application/json' \
    -d '{"notreal" : []}'

# example as query string parameter
curl -XPOST 'https://api.lytics.io/api/segment?access_token=804ef78pretendtoken8762' \
    -H 'Content-type: application/json' \
    -d '{"notreal" : []}'

Additionally, there are two types of authentication token’s:

  • User Auth Token is normally just for the web admin. But may be used on the api, this is a user-specific token, and attributes actions to this user. This token expires.

  • API User is a less privileged role and does not expire. But, less history is available on actions.

Documentation Examples

We use jq json command line prettifier in our examples throughout this doc.

Media Types

Our API is a JSON REST API. We have data-upload api’s which support csv uploads as well.

Requests with a message-body use plain JSON to set or update resource states.

Error States

The common HTTP Response Status Codes are used.

Query Parameters

A variety of places our api accepts query parameters that allow a list of values. The documentation will often say it allows []string or []int (meaning an array of strings, or integers). When this occurs, we allow a variety of formats to pass these.

  • ids=1234 convert this to []string{“123”}

  • ids=[123,456] convert this to []string{“123”,“456”}

  • ids=123,456 convert this to []string{“123”,“456”}

  • ids=123&ids=456 convert this to []string{“123”,“456”}

  • ids[]=123&ids[]=456 convert this to []string{“123”,“456”} Note that we alias ids[] = ids

Data Upload

APIs for collecting or uploading data. There are two main APIs, one for uploading Large files (bulk, millions of records) and another for sending real-time, or slightly batched events (Less than 10MB).

stream A stream name is like a table in database, each stream is a differnt Type of data. You assign a name to your data uploads to identify each stream/type. However, it should be consistent across uploads of same type of data.

Timestamp Fields When uploading historical, or event data you may have a time/date at which an event occured. Normal event uploads get an event timestamp at the server time but you can over-ride this behavior so the timestamp is defined in the event you upload. This is a _timestampfield parameter in api’s below. The formatting for this is very, very lenient with over 50 format’s supported.


Bulk CSV Upload

/collect/bulk/{stream}{?access_token,dryrun,timestamp_field,headers,delimiter,append,droperrors,filename}

Upload a large collection of data via csv, or JSON files.

Caution our csv parser is very strict, with escaping quotes etc being less lenient than many parsers you may have worked with. This is to ensure high quality data.

Error Handling Upon error, it will stop processing and return. So, we suggest trying uploading files with dryrun=true to verify your files will be accepted. If a few rows have issue you might try droperrors to see how many it is.

Parameters

stream
string (required)
Ex: user_db

The DataType aka Table of type of data being uploaded. See Data, Streams in web admin.

access_token
string (optional)

The Lytics DATA API key. Not the management key. Or use Authorization Header.

dryrun
boolean (optional)
Default: false
timestamp_field
string (optional)

The name of the column or field in file that contains event timestamp.

headers
string (optional)

CSV only, Comma delimited list of header columns for csv (if not first row)

delimiter
string (optional)

CSV only, if not the default comma, either t=tab, or | may be accepted

append
string (optional)

CSV only, JSON encoded object of additional name-value pairs to append to each record.

droperrors
boolean (optional)
Default: false
filename
string (optional)
Ex: "myfile.csv"

Just for record-keeping in our event - stream.

POST

Bulk CSV Upload

/collect/bulk/{stream}{?access_token,dryrun,timestamp_field,headers,delimiter,append,droperrors,filename}

Upload CSV files (may be gzipped). They MUST contain headers: either in first row (by default) OR in headers querystring field. If the file contains a timestamp field you want us to use see timestamp_field querystring parameter for how we extract it.

Note Domain this api is on a different domain https://bulk.lytics.io because the api.lytics.io domain is limited to 10MB file uploads.

# the "user_regdata" is the STREAM (ie Type) of data, if there 
# is multiple different types of data (userdata, emaildata, events) 
# they would have different streams.

# user data
export LIODATAKEY="YOURKEY"
curl -s -H "Authorization: $LIOKEY" \
  -H 'Content-type: application/csv' \
  --data-binary @users.csv \
  "https://bulk.lytics.io/collect/bulk/user_regdata?timestamp_field=register_date" \
 | jq '.'


# only do a dry-run to see if it is formatted correctly
curl -s "https://bulk.lytics.io/collect/bulk/user_regdata?access_token=$LIODATAKEY&timestamp_field=register_date&dryrun=true" \
    --data-binary @users.csv -H 'Content-type: application/csv' | jq '.'

# append fields to each row
#  where   append = urlencode({"source":"crm"})
#  often used when file-name contains information

curl -s "https://bulk.lytics.io/collect/bulk/user_regdata?append=%7B%22source%22%3A%22crm%22%7D&access_token=$LIODATAKEY" \
    --data-binary @users.csv -H 'Content-type: application/csv' | jq '.'


# full example uploading gzipped file

echo 'user_id,email,reg_date,item_count
7456,[email protected],"2012-10-17T17:29:39.738Z",82
1234,[email protected],"2012-12-11T19:53:31.547Z",82' > users.csv

gzip users.csv

curl -s "https://bulk.lytics.io/collect/bulk/user_regdata?access_token=$LIODATAKEY&timestamp_field=reg_date&dryrun=true&filename=users.csv" \
    --data-binary @users.csv.gz \
    -H 'Content-type: application/csv' | jq '.'

Requests

Copy All
Header
Content-Type: application/csv
Copy All
Body
name,id,registerd_date
jane,123,10/21/2004

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": 200,
    "message": "success",
    "data": {
        "message_count": 2,
        "rejected_count": 0,
        "content-type":"application/json",
        "droperrors": false,
        "dryrun": false,
        "timestamp_field": "event_created",
        "filename": "myfile.csv"
    }
}

Bulk JSON Upload

/collect/bulk/{streamname}{?access_token,dryrun,timestamp_field,filename}

JSON New Line Delimited file uploads. Each line contains a single record. The only limit is that nested array/objects are not allowed (see examples below).

Note Domain this api is on a different domain https://bulk.lytics.io because the api.lytics.io domain is limited to 10MB file uploads.

Parameters

streamname
string (required)
Ex: user_db

The DataType, or “Table” of type of data being uploaded

access_token
string (optional)

The Lytics DATA API key. Not the management key. Or use Authorization Header.

dryrun
boolean (optional)
Default: false
timestamp_field
string (optional)

The name of the column or field in file that contains event timestamp.

filename
string (optional)
Ex: "myfile.json"

Just for record-keeping in our event - stream.

POST

Bulk JSON Upload

/collect/bulk/{streamname}{?access_token,dryrun,timestamp_field,filename}

JSON formatting info.

This JSON is ok:

{"order_ids":[1,2,3]}

This is not valid because it has objects nested inside arrays:

{"orders":[{"id":1,"price":22},{"id":2,"price":23}]}

Requests

Copy All
Header
Content-Type: application/json
Copy All
Body
{"event":"register","date":"2014/04/05"}
{"event":"login","date":"2014/04/05"}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": 200,
    "message": "success",
    "data": {
        "message_count": 2,
        "rejected_count": 0,
        "content-type":"application/json",
        "droperrors": false,
        "dryrun": false,
        "timestamp_field": "event_created",
        "filename": "myfile.json"
    }
}

Data Upload

/collect/json/{stream}{?access_token,dryrun,timestamp_field,filename}

Upload event(s) to API. Unlike the bulk API above, this only accepts valid JSON (not new line delimitted) and it accepts arrays (lists) of events or a single event. Can be used to upload changes to user-attributes, or event records. Limited to 10MB.

# Example for multiple events

export LIODATAKEY="YOURKEY"
curl -s -H "Authorization: $LIODATAKEY" \
  -H 'Content-type: application/json' \
  -XPOST "https://api.lytics.io/collect/json/crm_events?timestamp_field=date" \
-d'
[
    {"event":"registered","date":"10/03/2013","user_id":"1234","tags":["a","b","c"]}
    , {"event":"purchased","date":"10/03/2013","user_id":"456","price":123.45,"product_id":"123456"}
]
' | jq '.'


# example single record

curl -s -XPOST "https://api.lytics.io/collect/json/user_db?access_token=$LIODATAKEY" \
    -H 'Content-type: application/json' -d'
{"user_id":"1234","tags":["a","b","c"]}

' | jq '.'

Parameters

stream
string (required)
Ex: user_db

The DataType, or “Table” of type of data being uploaded

access_token
string (required)

The Lytics DATA API key (not management key)

dryrun
bool (optional)
Default: false
timestamp_field
string (optional)

The name of the column or field in file that contains event timestamp.

filename
string (optional)
Ex: "myfile.json"

Just for record-keeping in our event - stream.

POST

Data JSON Upload

/collect/json/{stream}{?access_token,dryrun,timestamp_field,filename}

Requests

Copy All
Header
Content-Type: application/json
Copy All
Body
[{"event":"register","date":"2014/04/05"},
  {"event":"login","date"}]

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
  "status": 200,
  "message": "success",
  "data": {
    "message_count": 2,
    "rejected_count": 0,
    "content-type":"application/json",
    "droperrors": false,
    "dryrun": false,
    "timestamp_field": "event_created",
    "filename": "myfile.json"
  }
}

Segment

Segments resource are named logical expressions of users. These segments may logically be built using other segments as well. The segment api provides a list of segments built by both the admin, pre-defined segments automatically added to each account as well as custom ones created through the UI.

Segments represent a number of concepts:

  • Segment A list of users, users are members (or not) of a segment

  • Trigger As a user enters, or leaves a segment, this is a trigger. Triggers do not have a Size, only a history of events.

  • Goal A segment that also implies conversion. In Lytics App, there are Goals that show more prominently in a funnel type interface.

  • Dynamic Attribute A user belonging to a segment describes an attribute about that user. For instance IsPurchaser is both an attribute that can help personaliation, as well as a segment. We call these special segments Aspects at Lytics.

  • Metric A segment when used exclusively for the purposes of tracking conversions/metric.

The Segment resource has the following attributes:

field DataType Description
id string unique id, Lytics assigned
created Date Created date
updated Date Last Updated Date
kind string What Type of segment from list above [“aspect”,“goal”] or leave empty for “segment”.
slug string customer-assigned unique id (We auto assigned by slugifying Name if not provided).
name string Name of this segment
description string Long text description (optional)
is_public bool Is this segment available as part of user profile api (entity api)
segment_ql string Segment QL Logical Expression
tags []string List of tags, for organization purposes

To see a list of available fields available for segmentation rules see catalog api.


Segment List

/api/segment{?table,valid}

Get a list of all segments for an account. You might be more interested in /api/segment/sizes api below which is a more abreviated set of info including number of users in that segment.

Parameters

table
string (optional)
Ex: user

Table for segments to fetch, “user” by default, specify “content” for content collections (aka segments).

valid
string (optional)
Ex: all

True, False or “all”. Get all segments, only valid, or only invalid.

GET

Segment List

/api/segment{?table,valid}

Get list Segments . Segments can refer to any Table.

The two main tables you will be working with are User and Content.

# Grab all segments from "user" table

curl "$LIOAPI/api/segment" -s -H "Authorization: $LIOKEY" | jq '.'

# Grab all "Content Collections" (ney, segments)

curl "$LIOAPI/api/segment?table=content" -s -H "Authorization: $LIOKEY" | jq '.'


Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": [
        {
            "id": "7a7f76ff97ad0e3e96c162f74aee8f8f",
            "account_id": "4f7b525bdba058fc096757a6",
            "author_id": "user_12345",
            "kind":"segment",
            "segment_ql": "FILTER AND ( visits > 5, last_visit >= \"2015-04-01 00:00:00Z\", last_visit <  \"2015-04-02 00:00:00Z\")", 
            "created": "2014-08-04T21:18:20.124Z",
            "updated": "2014-08-04T21:18:20.124Z",
            "description": "Casual users have been active in the last 90 days and show no extreme behavioral patterns.",
            "is_public": true, 
            "name": "Recent Visits", 
            "slug_name": "recent_visits", 
            "tags": [
                "email_trigger"
            ]
        }
    ]
}

Segment

/api/segment/{id}

Get Single Segment By ID or slug

Parameters

id
string (required)
Ex: my_segment_slug

id, or slug of a segment, part of url path

GET

Segment Fetch

/api/segment/{id}

Fetch a segment by id or slug

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "id": "7a7f76ff97ad0e3e96c162f74aee8f8f", 
        "account_id": "4f7b525bdba058fc096757a6", 
        "author_id": "4f7b525bdba058fc096757a6",
        "kind":"segment",
        "segment_ql": "FILTER AND ( visits > 5, last_visit >= \"2015-04-01 00:00:00Z\", last_visit <  \"2015-04-02 00:00:00Z\")", 
        "created": "2014-08-04T21:18:20.124Z",
        "updated": "2014-08-04T21:18:20.124Z",
        "description": "Casual users have been active in the last 90 days and show no extreme behavioral patterns.",
        "name": "Recent Visits", 
        "slug_name": "recent_visits", 
        "tags": [
            "email_trigger"
        ]
    }
}
PUT

Segment Update

/api/segment/{id}

Update A segment.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "id": "7a7f76ff97ad0e3e96c162f74aee8f8f", 
        "account_id": "4f7b525bdba058fc096757a6", 
        "author_id": "4f7b525bdba058fc096757a6",
        "kind":"segment",
        "segment_ql": "FILTER AND ( visits > 5, last_visit >= \"2015-04-01 00:00:00Z\", last_visit <  \"2015-04-02 00:00:00Z\")", 
        "created": "2014-08-04T21:18:20.124Z",
        "updated": "2014-08-04T21:18:20.124Z",
        "description": "Casual users have been active in the last 90 days and show no extreme behavioral patterns.",
        "name": "Recent Visits", 
        "slug_name": "recent_visits", 
        "tags": [
            "email_trigger"
        ]
    }
}
DELETE

Segment Delete

/api/segment/{id}

Delete A segment.

Response 204

POST

Segment Create

/api/segment/{id}

Create A segment. See /api/segment/validate method for testing out Segmentation rules.

Pass a Segment QL logic expression to create a Segment.

# Plain Text api will extract the "our_test_activelist" as the "slug" and "name" of
# the segment
curl -s -XPOST "https://api.lytics.io/api/segment" \
    -H "Content-type: text/plain" \
    -H "Authorization: $LIOKEY" \
    -d'

FILTER AND (
    visits > 5,
    last_visit >= "now-30d",
    scores.momentum >  10
)
ALIAS our_test_activelist

' | jq '.'

# or try managing segments in files so you can check them in
# to Bitbucket or Github.   Also addresses field-
curl -s -XPOST 'https://api.lytics.io/api/segment' \
    -H 'Content-type: text/plain' \
    -H "Authorization: your_api_token" \
    -d @my_segment.json | jq '.'


# Json Api Allows a few more fields such as description
curl -s -XPOST "https://api.lytics.io/api/segment" \
    -H "Authorization: $LIOKEY" \
    -H "Content-type: application/json" \
    -d'
{
    "name":"Most Active Users",
    "segment_ql": "FILTER AND (visits > 5,last_visit >= \"now-30d\", scores.momentum >  10)",
    "slug_name":"our_test_activelist",
    "is_public": true,
    "description":"a description of what this is for",
    "tags":["email_personalization","product"]
}
' | jq '.'

# create a json file of a segment
curl -s -XPOST "$LIOAPI/api/segment" \
    -H "Authorization: $LIOKEY" \
    -H "Content-type: application/json" \
    -d @segment.json | jq '.'



# Content Collection (ney, "Segment")
curl -s -XPOST "https://api.lytics.io/api/segment" \
    -H "Content-type: text/plain" \
    -H "Authorization: $LIOKEY" \
    -d'

FILTER AND (
   EXISTS imageurls     -- Make sure they have images
   aspects = "article"  -- Ensure they are of type article
   PATH = "blog"        -- only show those from the /blog  part of site
   created > "now-30d"  -- only those authored in last 30 days
)
FROM content
WITH 
   name = "Recent Blog Articles With Images"
ALIAS recent_blog_articles

' | jq '.'



SegmentQL

The query language for segments

Filter     = "FILTER" Phrase [FROM] [ALIAS]
Phrase     = AND | OR | Expression
AND        = "AND" (Phrase, Phrase, ...)
OR         = "OR" (Phrase, Phrase, ...)
Expression = NOT
           | Comparison
           | EXISTS
           | IN
           | CONTAINS
           | LIKE
           | IncludeSegment
NOT            = "NOT" Phrase
Comparison     = Identifier ComparisonOp Literal
ComparisonOp   = ">" | ">=" | "<" | "<=" | "==" | "!="
EXISTS         = "EXISTS" Identifier
IN             = Identifier "IN" (Literal, Literal, ...)
CONTAINS       = Identifier "CONTAINS" Literal
LIKE           = Identifier "LIKE" String # uses * for wildcards
IncludeSegment = "INCLUDE" Identifier

# Optional From, not needed for segmentation on users
FROM          = "FROM" Identifier

# Alias for giving a segment a "name" which is how
#  it will be included from other filters
ALIAS         = "ALIAS" Identifier 

Literal = String | Int | Float | Bool | Timestamp

# note, identifiers may not contain spaces, or periods, etc
Identifier = [a-zA-Z][a-zA-Z0-9_]+

Examples

# Simple single expression filter
FILTER "abc" IN some_identifier

FILTER NOT foo

# Filters have an optional "Alias" used for 
# referencing 

FILTER AND ( channelsct > 1 AND scores.quantity > 20 ) ALIAS multi_channel_active


# Filters can references to other Filters
FILTER AND (
   EXISTS email,
   NOT INCLUDE multi_channel_active
)


FILTER AND (
    visits > 5,
    NOT INCLUDE someotherfilter,
)

# negation
FILTER NOT AND ( ... )

# Compound filter
FILTER AND (
    visits > 5,
    last_visit >= "2015-04-01 00:00:00Z",
    last_visit <  "2015-04-02 00:00:00Z",
)

# Like
FILTER url LIKE "/blog/"

# date math
# Operator is either + or -. Units supported are y (year), M (month), 
#   w (week), d (date), h (hour), m (minute), and s (second)

FILTER last_visit > "now-24h"

# IN
city IN ("Portland, OR", "Seattle, WA", "Newark, NJ")

# Complex
AND (
    OR (
        foo == true
        bar != 5
    )
    EXISTS signup_date
    OR (
        NOT bar IN (1, 2, 4, 5)
        INCLUDE SomeOtherFilter
    )
)

# Time Windows
# Can only be used on timebucket fields
# Answers the question "at least X in Y days", where X can be one of (1, 5, 10, 25, 50, 100)
# and Y can be one of (7, 30) days.
#
# timewindow(fieldname, X, Y)
FILTER timewindow(bucketfield, 5, 7)


# Between
FILTER AND (
    _modified BETWEEN "2015-07-01" AND "2016-08-01"
)


Response 201

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "id": "7a7f76ff97ad0e3e96c162f74aee8f8f", 
        "account_id": "4f7b525bdba058fc096757a6", 
        "author_id": "4f7b525bdba058fc096757a6",
        "kind":"segment",
        "segment_ql": "FILTER AND ( visits > 5, last_visit >= \"2015-04-01 00:00:00Z\", last_visit <  \"2015-04-02 00:00:00Z\")", 
        "created": "2014-08-04T21:18:20.124Z",
        "updated": "2014-08-04T21:18:20.124Z",
        "description": "Casual users have been active in the last 90 days and show no extreme behavioral patterns.",
        "name": "Recent Visits", 
        "slug_name": "recent_visits", 
        "tags": [
            "email_trigger"
        ]
    }
}

Segment Size

/api/segment/size

Get Segment Size for a Segment Statement.

POST

Segment Size

/api/segment/size

Get size of a single Segment using SegmentQL, not saved.

This api is very rate throttled, 110 seconds.

# Post a SegmentQL query to size api to fetch size
curl -s -XPOST "https://api.lytics.io/api/segment/size" \
    -H "Content-type: text/plain" \
    -H "Authorization: your_api_token" \
    -d'

FILTER AND (
    visits > 5,
    last_visit >= "now-30d",
    scores.momentum >  10
)

' | jq '.'

Requests

Copy All
Header
Content-Type: text/plain
Copy All
Body
FILTER AND (
    visitct > 10,
    EXISTS email,
    scores.momentum > 25
)

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "id":"",
        "name":"active",
        "slug_name":"active",
        "size":10402
    }
}

Segment Validate

/api/segment/validate

Validate single Segment Query Statement

# Plain Text api for testing segments
curl -s -XPOST 'https://api.lytics.io/api/segment/validate' \
    -H 'Content-type: text/plain' \
    -H "Authorization: your_api_token" \
    -d'

FILTER AND (
    visits > 5,
    last_visit >= "now-30d",
    scores.momentum >  10
)

' | jq '.'
POST

Segment Validate

/api/segment/validate

Validate single Segment Query Statement

Requests

Copy All
Header
Content-Type: text/plain
Copy All
Body
FILTER AND (
    visitct > 10,
    EXISTS email,
    scores.momentum > 25
)

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": 200,
    "message":"success",
    "data": null
}

Segment Sizes

/api/segment/sizes{?ids}

Get Segment Sizes. Pass a list of ids, or none for ALL for this account.

Parameters

ids
[]string (required)
Ex: 123,456

list of ids to get, format = ids[]=123&ids[]=234 OR ids=123,345 etc, see standard []string param doc

GET

Segment Sizes

/api/segment/sizes{?ids}

Get bunch of segment sizes.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data" : [
        {
            "id":"456dert",
            "name":"Active",
            "slug_name":"active",
            "size":10402
        },
        {
            "id":"1234abcd",
            "type":"custom",
            "slug_name":"active",
            "size":45236
        }
    ]
}

Segment Scan

/api/segment/{segId}/scan{?id,limit,splits,start,meta,sortfield,sortorder,recent,segments,fields}

Get a list of all users (or other entity types) for a segment.

Parameters

segId
string (optional)
Ex: 1234

segment Id in path

id
string (optional)
Ex: 1234

segment Id in querystring instead of path then use api/segment/scan

limit
integer (optional)
Ex: 20

num of rows to return per call, default = 20, max = 1000

splits
[]integer (optional)
Ex: 1,2,3

1,2,3 comma seperated list of ints 0-99 for a-b split testing (a=1,5,8 )

start
integer (optional)
Ex: next_xyz

optional ID of a previous segment scan to resume, must be recent this is retrieved from the next parameter in JSON response

meta
boolean (optional)
Ex: true

optional boolean, whether to include schema in response

sortfield
string (optional)
Ex: last_visit_ts

field to sort on

sortorder
string (optional)
Ex: 20

[desc,asc]

recent
boolean (optional)
Ex: true

short cut for last visit timestamp desc [t,true,]

segments
string (optional)
Ex: {rules...}

JSON of ad-hoc segment, or pass this in http Body

fields
[]string (optional)
Ex: email,user_id,name

list of fields to include in response

GET

Segment Scan

/api/segment/{segId}/scan{?id,limit,splits,start,meta,sortfield,sortorder,recent,segments,fields}

Get a paged list of Users for a Segment.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "next": "biglongstring",
    "data": [
        {
            "email": "[email protected]", 
            "user_id": 12, 
            "visitct": 100
        },
        ...
    ]
}

Segment Collection

Segment Collections are grouped/named lists of segments. Segments that participate in a collection such as Goals are related. The Lytics App ui has one predefined collection called Goals which defines the main funnel of user conversions for your company. Definining this helps our Machine Learning understand goals.

In addition, the Lytics App has a variety of collections for Aspects on the home page/reporting which are collections of segments on related themes.

  • Email Collection of aspects related to email behavior.

  • Web A variety of aspects of web users.

  • Ecommerce


Segment Collection List

/api/segmentcollection

Get a list of all segment collections.

GET

Segment Collection List

/api/segmentcollection

Get list Segment Collections

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": [
        {
          "aid": 12,
          "account_id": "4f7b789bdba123fc096757a6",
          "id": "c6f8341eecc4a0aa1ff12cd7f493445c",
          "name": "Goal Funnel",
          "slug_name": "goal_funnel",
          "table": "user",
          "author_id": "1bb376fe32bfa9db34a18b481af2156d",
          "updated": "2016-05-16T22:33:23.699Z",
          "created": "2016-01-08T00:10:20.748Z",
          "internal": false,
          "collection": [
            {
              "id": "2cdf46b7afad96f3a4e4b41cee40ecd6",
              "order": 0
            },
            {
              "id": "a3c8185e14282015039e5ab5b05a0da1",
              "order": 1
            },
            {
              "id": "bdef1f39c6b9cbc46712a836512888d7",
              "order": 2
            },
            {
              "id": "8d469110b2d351c2dcbb36bfb6313c20",
              "order": 3
            }
          ],
          "parent_segment": ""
        }
    ],
    "message": "success",
    "status": 200
}

Segment Collection

/api/segmentcollection/{id}

Get Single segment collection By ID

Parameters

id
string (required)
Ex: 1234abcd

id of segment collection

GET

Segment Collection Fetch

/api/segmentcollection/{id}

Fetch a segment collection by id or slug

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data":  {
          "aid": 12,
          "account_id": "4f7b789bdba123fc096757a6",
          "id": "c6f8341eecc4a0aa1ff12cd7f493445c",
          "name": "Goal Funnel",
          "slug_name": "goal_funnel",
          "table": "user",
          "author_id": "1bb376fe32bfa9db34a18b481af2156d",
          "updated": "2016-05-16T22:33:23.699Z",
          "created": "2016-01-08T00:10:20.748Z",
          "internal": false,
          "collection": [
            {
              "id": "2cdf46b7afad96f3a4e4b41cee40ecd6",
              "order": 0
            },
            {
              "id": "a3c8185e14282015039e5ab5b05a0da1",
              "order": 1
            },
            {
              "id": "bdef1f39c6b9cbc46712a836512888d7",
              "order": 2
            },
            {
              "id": "8d469110b2d351c2dcbb36bfb6313c20",
              "order": 3
            }
          ],
          "parent_segment": ""
    }
}
POST

Segment Collection Create

/api/segmentcollection/{id}

Create a new Segment Collection from existing segments.

curl -s -J -XPOST "https://api.lytics.io/api/segmentcollection" -H "Authorization: $LIOKEY" -d '
{
    "name": "My Collection",
    "collection":[
        {"id":"1234abcd", "order": 0}
        , {"id":"3456abcd", "order": 1}
    ]
}
'

Requests

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "name": "Collection Name",
    "collection": [
            {"id":"1234abcd", "order": 0}
            , {"id":"3456abcd", "order": 1}
    ]
}

Response 201

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data":  {
          "aid": 12,
          "account_id": "4f7b789bdba123fc096757a6",
          "id": "c6f8341eecc4a0aa1ff12cd7f493445c",
          "name": "Goal Funnel",
          "slug_name": "goal_funnel",
          "table": "user",
          "author_id": "1bb376fe32bfa9db34a18b481af2156d",
          "updated": "2016-05-16T22:33:23.699Z",
          "created": "2016-01-08T00:10:20.748Z",
          "internal": false,
          "collection": [
            {
              "id": "2cdf46b7afad96f3a4e4b41cee40ecd6",
              "order": 0
            },
            {
              "id": "a3c8185e14282015039e5ab5b05a0da1",
              "order": 1
            },
            {
              "id": "bdef1f39c6b9cbc46712a836512888d7",
              "order": 2
            },
            {
              "id": "8d469110b2d351c2dcbb36bfb6313c20",
              "order": 3
            }
          ],
          "parent_segment": ""
    }
}
PUT

Segment Collection Update

/api/segmentcollection/{id}

Update a Segment Collection

curl -s -J -XPUT "https://api.lytics.io/api/segmentcollection/1234abcd" -H "Authorization: $LIOKEY" -d '
{
    "name": "My Collection",
    "collection":[
        {"id":"1234abcd", "order": 0}
        , {"id":"3456abcd", "order": 1}
    ]
}
'

Requests

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "name": "Collection Name",
    "collection": [
            {"id":"1234abcd", "order": 0}
            , {"id":"3456abcd", "order": 1}
    ]
}

Response 201

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data":  {
          "aid": 12,
          "account_id": "4f7b789bdba123fc096757a6",
          "id": "c6f8341eecc4a0aa1ff12cd7f493445c",
          "name": "Goal Funnel",
          "slug_name": "goal_funnel",
          "table": "user",
          "author_id": "1bb376fe32bfa9db34a18b481af2156d",
          "updated": "2016-05-16T22:33:23.699Z",
          "created": "2016-01-08T00:10:20.748Z",
          "internal": false,
          "collection": [
            {
              "id": "2cdf46b7afad96f3a4e4b41cee40ecd6",
              "order": 0
            },
            {
              "id": "a3c8185e14282015039e5ab5b05a0da1",
              "order": 1
            },
            {
              "id": "bdef1f39c6b9cbc46712a836512888d7",
              "order": 2
            },
            {
              "id": "8d469110b2d351c2dcbb36bfb6313c20",
              "order": 3
            }
          ],
          "parent_segment": ""
    }
}
DELETE

Segment Collection Delete

/api/segmentcollection/{id}

Delete a Segment Collection model.

Parameters

id
string (required)
Ex: source::target

ID of the Segment Collection model to delete.

Response 204


Subscription

BETA API

Subscriptions are queries into real-time events in Lytics, most commonly listening to a list of users entering/leaving segments ie triggers. Upon Entering/Leaving you want to send a notification callback with that user data so you can email them, save it, etc.

Channels Types of streams the event can be delivered.

  • Webhooks

  • AWS Kinesis

  • Http Streaming

  • ??? Ask us Azure Event-Hub, Google PubSub.

Some features:

  • withbackfill this true/false flag to indicate whether to backfill webhook events for users currently a member of a segment. Default is false, so only new users entering/exiting a segment would cause a webhook. If you enter withbackfill=true then all users currently a member of the segment at time of submission would have a webhook fired for them as well, then changes going forward would start firing. If backfill is used, only a single segment can be specified.

Example

export LIOKEY="your_api_key"
export LIODATAKEY="your_data_key"

# UPSERT Segment uses "ALIAS" as id for create/update
curl -s -H "Authorization: $LIOKEY" -H "Content-Type: text/plain" \
  -XPOST "https://api.lytics.io/api/segment" -d '

-- Paying Users Segment
FILTER AND (
    paying_user = "t"
)
FROM user
ALIAS demo_segment

' | jq '.'

# get that Segment ID
SEGID=`curl -s "https://api.lytics.io/api/segment?key=$LIOKEY" | jq -r '.data[] | select(.name == "demo_segment") | .id'`
echo "Created Segment id=$SEGID"

# Create Webhook Subscription to send entry/exits events to your url
# - entry/exits will be sent to the ngrok url below (create your own)
# - we use app-engine for this
echo '
{
    "webhook_url": "http://123456abc.ngrok.com/"
    , "segment_ids": ["1234abcd"]
    , "withbackfill": false
    , "vesion" : "new"
    , "channel" : "webhook"
    , "name" : "webhooktest1"
    , "description": "What is this subscription for?"
}
' |  \
curl -v -XPOST "https://api.lytics.io/api/subscription" \
   -H "Authorization: $LIOKEY" \
   -H "Content-Type: application/json" -d @- | jq '.'

# for kinesis

curl -s -H "Authorization: $LIOKEY" -H "Content-Type: application/json" \
  -XPOST "https://api.lytics.io/api/subscription" -d '
{
    "channel":"kinesis",
    "segment_ids": ["1234abcd","abc345"],
    "name":"test2",
    "description":"hello world",
    "stream":"lytics_triggers_stream",
    "region": "us-east-1",
    "identifier_field":"user_id",
    "role_arn":"arn:aws:iam::4444444444444:role/LyticsWriteToKinesis"
}
' | jq "."

# send test data ...

# toggling these next two should move user in/out of segment causing trigger
curl -s -XPOST "https://api.lytics.io/collect/json/demo" -H "Authorization: $LIODATAKEY"  -H 'Content-type: application/json' \
  -d '{ "paying_user":"f", "user_id":"user123"}' | jq '.'

curl -s -XPOST "https://api.lytics.io/collect/json/demo" -H "Authorization: $LIODATAKEY"  -H 'Content-type: application/json' \
  -d '{ "paying_user":"t", "user_id":"user123"}' | jq '.'

Webhook Post Example This will send a json document like below to your subscription channel. But, this is a BETA API and this will change. We received feedback that the more logical set of data is all segment changes (not just one per subscription) so we are modifying the response to be an array of segment entry/exits.

Example Message

There are still message format from v1 of this api to maintain backwards compatibility.

{
  "data": {
    "_created": "2016-06-29T18:50:16.902758229Z",
    "_modified": "2017-03-18T06:12:36.829070108Z",
    "email": "[email protected]",
    "user_id": "user123",
    "segment_events":[
        {
            "id": "d3d8f15855b6b067709577342fe72db9",
            "event": "exit",
            "enter": "2017-03-02T06:12:36.829070108Z",
            "exit": "2017-03-18T06:12:36.829070108Z",
            "slug": "demo_segment"
        },
        {
            "id": "abc678asdf",
            "event": "enter",
            "enter": "2017-03-02T06:12:36.829070108Z",
            "exit": "2099-03-18T06:12:36.829070108Z",
            "slug": "another_segment"
        }
    ]
  },
  "meta":{
     "object":"user",
     "source":"subscription",
     "subscription_id": "7e2b8804bbe162cd3f9c0c5991bf3078"
  }
}

DEPRECATED

There are still message format from v1 of this api to maintain backwards compatibility. This will be deprecated in June 2017.

{
  "data": {
    "_audience": "241b1530768bf521d0dc671cbee981bd",
    "_audience_action": "entered",
    "_audience_friendly": "demo_segment",
    "_created": "2016-06-29T18:45:53.90305967Z",
    "_modified": "2016-07-27T00:11:26.960853872Z",
    ...
    <attributes snipped, your users will be specific to your account>
    ...
    "paying_user": "t",
    }
  },
  "segment.entered": "2016-07-27T00:11:28.353480509Z",
  "subscriptionid": "07a8d53f0709a31d758fe6bdaa067a49",
  "object": "segment",
  "objectid": "241b7510368bf521d0dc671cbee981bd"
}

Subscription List

/api/subscription

Get all current subscriptions.

GET

Subscription List

/api/subscription

Get all Subscriptions.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": [
        {
            "id": "3573009ab39bbc9614f754c0fafd53d7",
            "description": "hello world",
            "segment_ids": [
                "f897270369f977294871f21cc0d826e7"
            ],
            "segmentql": "FILTER AND ( score_momentum \u003e 25, score_quantity \u003e 25 ) ALIAS seg_webhook1",
            "webhook_url": "http://ourtestwebhook.appspot.com/capture",
            "description": "hello world",
            "name": "test1",
            "updated": "2016-06-30T15:33:14.376262062-07:00",
            "created": "2016-06-30T15:33:14.37626196-07:00",
            "user_id": "2916b84ba4fa8b510591fd6e1c2320db",
            "account_id": "testid",
            "workflow": "webhook_triggers"
        }
    ]
}

Subscription

/api/subscription/{id}

Get a single Subscription resource.

Parameters

id
string (required)
Ex: 1234

subscription Id in path

GET

Subscription Fetch

/api/subscription/{id}

Get a single Subscription Webhook.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "id": "3573009ab39bbc9614f754c0fafd53d7",
        "description": "hello world",
        "segment_ids": [
            "f897270369f977294871f21cc0d826e7"
        ],
        "segmentql": "FILTER AND ( score_momentum \u003e 25, score_quantity \u003e 25 ) ALIAS seg_webhook1",
        "webhook_url": "http://ourtestwebhook.appspot.com/capture",
        "description": "hello world",
        "name": "test1",
        "updated": "2016-06-30T15:33:14.376262062-07:00",
        "created": "2016-06-30T15:33:14.37626196-07:00",
        "user_id": "2916b84ba4fa8b510591fd6e1c2320db",
        "account_id": "testid",
        "workflow": "webhook_triggers"
    }
}
POST

Subscription Upsert

/api/subscription/{id}

Upsert a new Subscription. Subscriptions allow you to listen and recieve notification of events inside of Lytics, most common would be listen to users entering/leaving segments to serve as triggers.

When creating subscriptions they have the following fields.

field DataType Description
channel string Required: What Type of subscription. [kinesis, webhook]
name string Name of this Subscription
description string Long text description (optional)
segment_ids []string List of segment Ids
withbackfill bool Should all members of this segment be included? They will be marked as “entered” segment.

Webhook only fields | field | DataType | Description | |—————|——————|—————| | webhook_url | string | Required for Webhooks, only used on webhooks, url to post to.

Kinesis only fields | field | DataType | Description | |——————|——————|—————| | region | string | Required. AWS Region | stream | string | Required. Kinesis stream | role_arn | string | AWS IAM Role to use for assume-role (see integrations doc) | identifier_field | string | Optional. If provided, pull this field from user field and use as shard-key in Kinesis Put.

Create a Webhook Subscription that includes the Segment definition and custom headers (optional)


curl -s -H "Authorization: $LIOKEY" -H "Content-Type: application/json" \
  -XPOST "https://api.lytics.io/api/subscription" -d '
{
    "segmentql": "FILTER AND ( score_momentum > 25, score_quantity > 25 ) ALIAS seg_webhook1",
    "name":"test1",
    "channel" : "webhook",
    "description":"hello world",
    "webhook_url":"https://lyticswwwapp.appspot.com/log",
    "headers": {"Authorization": "dGhpczp0aGF0"}
}
' | jq "."

Create a Webhook subscription with list of SegmentId’s (previously created) to listen to.

curl -s -H "Authorization: $LIOKEY" -H "Content-Type: application/json" \
  -XPOST "https://api.lytics.io/api/subscription" -d '
{
    "segment_ids": ["1234abcd","abc345"],
    "channel":"webhook",
    "name":"test2",
    "description":"hello world",
    "webhook_url":"https://lyticswwwapp.appspot.com/log"
}
' | jq "."

Create a Kinesis Subscription


curl -s -H "Authorization: $LIOKEY" -H "Content-Type: application/json" \
  -XPOST "https://api.lytics.io/api/subscription" -d '
{
    "channel":"kinesis",
    "segment_ids": ["1234abcd","abc345"],
    "name":"test2",
    "description":"hello world",
    "stream":"lytics_triggers_stream",
    "region": "us-east-1",
    "identifier_field":"user_id",
    "role_arn":"arn:aws:iam::4444444444444:role/LyticsWriteToKinesis"
}
' | jq "."



Requests

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "segment_ids": ["1234abcd","abc345"],
    "name":"test2",
    "channel": "webhook",
    "withbackfill":false,
    "description":"hello world",
    "webhook_url":"https://lyticswwwapp.appspot.com/log"
}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "id": "3573009ab39bbc9614f754c0fafd53d7",
        "description": "hello world",
        "segment_ids": [
            "f897270369f977294871f21cc0d826e7"
        ],
        "segmentql": "FILTER AND ( score_momentum \u003e 25, score_quantity \u003e 25 ) ALIAS seg_webhook1",
        "webhook_url": "http://ourtestwebhook.appspot.com/capture",
        "description": "hello world",
        "name": "test1",
        "updated": "2016-06-30T15:33:14.376262062-07:00",
        "created": "2016-06-30T15:33:14.37626196-07:00",
        "user_id": "2916b84ba4fa8b510591fd6e1c2320db",
        "account_id": "testid",
        "workflow": "webhook_triggers"
    }
}

Subscription Streaming

/api/subscription/streaming{segment_id}

Create a temporary subscription to view subscriptions as long as the connection is open. Once the http connection is closed the subscription stops.


# Test api fake events just to see if streaming connection is working

curl -s --max-time 3600 --no-buffer -H "Authorization: $LIOKEY" \
   https://api.lytics.io/api/subscription/streaming_test | jq '.'


# subscribe to all entry/exit events for given SEGID

curl -s --max-time 3600 --no-buffer -H "Authorization: $LIOKEY" \
  "https://api.lytics.io/api/subscription/streaming?segment_id=$SEGID" | jq "."

Parameters

segment_id
string (required)
Ex: abc123

segmentid to listen for events


Personalization

Entity API, ie Personalization or Profile API. Retrieve attributes and segments a user is a member of for a User (Entity).

The Node is a User most likely, but since Lytics is organized as a Graph Database, other types of Nodes may be modeled such as Accounts, or Company, etc..


Personalization

/api/entity/user/{fieldname}{fieldval}{?fields,segments,meta}

Used to return the Entity object (usually a user, content item, url, account, etc).

  • :entitytype user, account, content, product etc

Parameters

fieldname
string (required)
Ex: email

Field name of identity to use as lookup

fieldval
string (required)
Ex: [email protected]

Field value of identity

fields
string (optional)
Ex: name,org,city

comma delimited field list to be returned

segments
bool (optional)
Default: true

Should segments this user is member of be included?

meta
bool (optional)
Default: true

Should Meta Fields be included?

GET

User Me

/api/entity/user/{fieldname}{fieldval}{?fields,segments,meta}

Get single User Entity. This is an alias for /api/entity/user.

Parameters

fieldname
string (required)
Ex: email

Field name of identity to use as lookup

fieldval
string (required)
Ex: [email protected]

Field value of identity

fields
string (optional)
Ex: name,org,city

comma delimited field list to be returned

segments
bool (optional)
Default: true

Should segments this user is member of be included?

meta
bool (optional)
Default: true

Should Meta Fields be included?

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "segments": [
            "email_us_based",
            "web_users",
            "all"
        ]
    },
    "message": "success",
    "meta": {
        "by_fields": [
            "user_id",
            "email",
            "fbuid",
            "_uids"
        ],
        "format": "json",
        "name": "user"
    },
    "status": 200
}
GET

Personalization

/api/entity/user/{fieldname}{fieldval}{?fields,segments,meta}

Get single User Entity.

Parameters

fieldname
string (optional)
Default: _uid
fieldval
string (required)
Ex: [email protected]

Field value of identity

fields
string (optional)
Ex: name,org,city

comma delimited field list to be returned

segments
bool (optional)
Default: true

Should segments this user is member of be included?

meta
bool (optional)
Default: true

Should Meta Fields be included?

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "segments": [
            "email_us_based",
            "web_users",
            "all"
        ]
    },
    "message": "success",
    "meta": {
        "by_fields": [
            "user_id",
            "email",
            "fbuid",
            "_uids"
        ],
        "format": "json",
        "name": "user"
    },
    "status": 200
}
GET

Web Personalization

/api/entity/user/{fieldname}{fieldval}{?fields,segments,meta}

Get single User Entity using cookie id.

Parameters

fieldval
string (required)
Ex: [email protected]

Field value of identity

fields
string (optional)
Ex: name,org,city

comma delimited field list to be returned

segments
bool (optional)
Default: true

Should segments this user is member of be included?

meta
bool (optional)
Default: true

Should Meta Fields be included?

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "segments": [
            "email_us_based",
            "web_users",
            "all"
        ]
    },
    "message": "success",
    "meta": {
        "by_fields": [
            "user_id",
            "email",
            "fbuid",
            "_uids"
        ],
        "format": "json",
        "name": "user"
    },
    "status": 200
}

Content

Content Api’s for Lytics Content Affinity Engine, allow for adding content into lytics, getting lists of content, and understanding the content and content-topics.

Content Classification Content classification refers to the Lytics automated process of taking text, url, image based content and classifying it to attach topics to content documents.

  • Content A list of content (Urls, Email Documents, Uploaded Documents, Campaigns) and the topics, created dates, etc for each.

  • Content Topics Categories/Topics are applied to each piece of content.

  • User Content Affinity Based on user interacting with content, the topics (derived from the content they interact with) is enriched into their profile.

First, an overview of How this content gets here.

  • Observed Urls Once the lytics tag is on a Website, as users use the website any new url they interact with is collected.

  • Crawling we can proactively crawl a site.

  • Email Tools We see urls inside of email campaigns and proactively classify those url’s, as well as the emails themselves.

  • Uploaded Content You may upload content to the Classify.

Use Cases

1) User Content Recomendations Find content for a user based on their content-affinity.

2) Get List of Content Rich query api to get content.

Examples


# ad-hoc segment scan from content table
# get a list of content documents that
#  - have aspects=article   (aspects = article,product, etc)
#  - has imageurl
curl -s -H "Authorization: $LIOKEY" \
 -XGET "https://api.lytics.io/api/segment/scan?sortfield=created&limit=10" -d '
FILTER AND (
 EXISTS imageurls
 aspects = "article"
)
FROM content
' | jq '.'

# Create a content-segment 
# - uses "ALIAS" as id for create/update, must be unique
curl -s -H "Authorization: $LIOKEY" -H "Content-Type: text/plain" \
  -XPOST "$LIOAPI/api/segment" -d '
FILTER AND (
 EXISTS imageurls
 aspects = "article"
)
FROM content
WITH name = "My Articles"
ALIAS my_articles
' | jq '.'

# grab just those content-docs using named segment scan
curl -s -H "Authorization: $LIOKEY" \
 -XGET "https://api.lytics.io/api/segment/my_articles/scan?sortfield=created&limit=10" \
  | jq '.'


# grab recommended articles for single user

curl -s -H "Authorization: $LIODATAKEY" \
 -XGET "https://api.lytics.io/api/content/recommend/:your_account_id/user/user_id/123456" \
  | jq '.'


Content Topics

/api/content/topic

Get a list of topics in the content graph.

GET

Content Topics

/api/content/topic

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
  "data": [
    {
      "name": "marketing technology",
      "doc_count": 2
    },
    {
      "name": "AdWords",
      "doc_count": 1
    },
    {
      "name": "ad retargeting",
      "doc_count": 4
    },
    {
      "name": "analytics",
      "doc_count": 2
    },
    {
      "name": "customer service",
      "doc_count": 1
    },
    {
      "name": "data visualization",
      "doc_count": 2
    },
    {
      "name": "data-first marketing",
      "doc_count": 157
    },
    {
      "name": "personalization",
      "doc_count": 15
    }
  ]
}

Content Topic Summary

/api/content/topic/{topic}

Get a summary of user affinity for, and related documents to a topic.

For each given topic, find out how many users have a high, low, none affinity for a given topic.

GET

Content Topic Summary

/api/content/topic/{topic}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": 200,
    "message": "success",
    "data": {
       "topics": {
            "total": 3232717,
            "missing": 2985477,
            "present": 239239,
            "bucket_none": 8001,
            "bucket_low": 26279,
            "bucket_mid": 11742,
            "bucket_high": 201218,
            "avg": 0.90787879293533
        },
        "docs": {
            "total": 373,
            "urls": [
                {
                    "url": "www.our-site.com/product/new-product-xyz",
                    "title": "Blue Running Shoes that make you faster",
                    "topics": [
                        "our-site (company)",
                        "running",
                        "shoes"
                    ],
                    "id": "3395466611293777389"
                }
            ]
        }
    }
}

Content Topic Rollups

/api/content/rollup/{label}

High-level categories can be expressed as Topic Rollups.
Topic rollups are user defined, and can simplify your interactions with using content for personalization and recommendation.
Topics and rollups share a many-to-many relationships — topic rollups should have multiple low-level topics and low-level topics can subscribe to multiple topic rollups.

GET

Content Topic Rollup Summary

/api/content/rollup/{label}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "docs": {
            "total": 2,
            "urls": [
                {
                    "author": "Cookie Monster",
                    "created": "2016-01-14T08:39:01.354824186-07:00",
                    "description": "you should definitely make these cookies",
                    "fetched": "2016-07-19T13:39:20.158747984-04:00",
                    "id": "4525087172572165639",
                    "meta": [
                        "og:locale/en_US"
                    ],
                    "primary_image": "http://cookdiary.net/wp-content/uploads/images/Oatmeal-Chocolate-Chip-Cookies_1441.jpg",
                    "stream": "default",
                    "title": "how to make some cookies",
                    "topic_relevances": {
                        "Recipe": 0.64,
                        "baking": 0.75
                    },
                    "topic_rollups": {
                        "Things": 0.75
                    },
                    "topics": [
                        "baking",
                        "Recipe"
                    ],
                    "updated": "2016-07-19T13:34:18-04:00",
                    "url": "www.recipes.com/chocolate-chip-cookies"
                },
                {
                    "author": "The Swedish Chef",
                    "created": "2016-07-19T13:39:20.158747984-04:00",
                    "description": "a delicious cake",
                    "id": "-2119104978705508335",
                    "primary_image": "http://www.vanillacake.com/cake.jpg",
                    "stream": "default",
                    "title": "Basic Vanilla Cake Recipe",
                    "topic_relevances": {
                        "Recipe": 0.64,
                        "baking": 0.85
                    },
                    "topic_rollups": {
                        "Desserts": 0.85
                    },
                    "topics": [
                        "Recipe",
                        "baking"
                    ],
                    "updated": "2016-07-19T13:29:18-04:00",
                    "url": "www.recipes.com/cake"
                }
            ]
        },
        "rollups": null
    },
    "message": "success",
    "status": 200
}

Content Document

/api/content/doc{?urls,ids}

Get a summary for document. If no hashed url is specified, then return the 10 most recent urls. Summaries may include the following properties:

Property Description
url External location of the content source
title Contents of the title tag in the case of a URL. In the case of email, the contents of the subject line.
description Contents of the meta tag with the name property of description, og:description, or twitter:description.
topics A set of strings identifying relevant topics in the content. Each topic can be mapped directly to a Wikidata entity.
topic_relevances An object containing each topic and its predicted relevance.
topic_rollups An object containing each topic rollup and its composite relevance.
primary_image URL of the primary image in the content.
author Plain-text name of the author.
created Date of publication. If unavailable, defaults to the time it is fetched by Lytics.
id Reverse domain name notation of the canonicalized value of the URL.
sitename Plain-text name of the site.
stream Name of the Lytics stream where the URL was observed.
path Path of the URL.
aspects Contextual type of a URL. One of article, discussion, image, product, video, other.
language Human language of the content.
updated Date the content was last updated. This usually corresponds to its enrichment date inside of Lytics.
fetched Date when the content was retrieved by Lytics.
hashedurl Hashed value of the input URL, used for identification within Lytics.

Parameters

urls
[]string (optional)
Ex: http//www.getlytics.com/blog/post

url to look up

ids
[]int64 (optional)
Ex: 123abc

hashed url or campaign id to look up

GET

Content Document

/api/content/doc{?urls,ids}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": 200,
    "message": "success",
    "data": {
        "total": 1,
        "urls": [
            {
                  "url": "www.our-site.com/product/new-product-xyz",
                  "title": "Blue Running Shoes that make you faster",
                  "topics": [
                      "our-site (company)",
                      "running",
                      "shoes"
                  ],
                  "id": "3395466611293777389"
            }
        ]
    }
}

Content Classify

/api/content/doc/classify{?url,text}

Classify the given url or text. This will enrich the given url/text to add tags to it, and add it to the content index.

The Data Upload api’s would allow you to relate user’s to these documents.

Parameters

url
string (optional)
Ex: http://myurl

url to classify

text
string (optional)
Ex: my text

text to classify

POST

Content Classify

/api/content/doc/classify{?url,text}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{  
  "data":{  
    "id":"https://www.getlytics.com/blog/post/improving_google_analytics?utm_source=linkedin",
    "url":{  
      "Params":{ },
      "Utm":{  
        "utm_campaign":null,
        "utm_content":null,
        "utm_medium":null,
        "utm_source":"linkedin",
        "utm_term":null
      },
      "UserInfo":null
    },
    "url_redirected_from":"",
    "description":"Stop Pretending Your Insights Are Actionable: Part 1, Google Analytics",
    "long_description":"Most analytics tools are used purely for insight discovery rather than empowering action. We explore how the Lytics analytics integrations focus on action and filling the some massive gaps traditional analytics tools miss.",
    "type":"web",
    "stream":"",
    "httpstatus":0,
    "humanLanguage":"en",
    "tags":[  
      {  
        "type":"meta",
        "id":"og:title",
        "value":"stop pretending your insights are actionable: part 1, google analytics",
        "source":"meta",
        "vocab":"lytics",
        "relevance":1
      },
      {  
        "type":"concept",
        "id":"Google_Analytics",
        "tax":[  
          "http://dbpedia.org/ontology/Company",
          "http://dbpedia.org/ontology/Website",
          "http://dbpedia.org/ontology/Work",
          "http://www.w3.org/2002/07/owl#Thing"
        ],
        "value":"Google Analytics",
        "source":"diffbot",
        "vocab":"dbpedia",
        "relevance":0.95
      }
    ],
    "flags":null,
    "images":[  
      {  
        "url":"https://www.getlytics.com/img/blog/defaults/developers.gif"
      }
    ],
    "primary_image":"https://www.getlytics.com/img/blog/defaults/developers.gif",
    "videos":null,
    "primary_video":"",
    "published":"2015-10-23T00:00:00Z",
    "author":"Mark Hayden",
    "numpages":0,
    "sitename":"Lytics",
    "productDescription":"",
    "brand":"",
    "price":"",
    "saveAmount":"",
    "productId":"",
    "created":"2016-06-24T21:20:43.421892991Z",
    "fetched":"2016-06-24T21:20:45.751844316Z",
    "updated":"2016-06-24T21:20:52.367247963Z",
    "enriched":{  
      "diffbot":"2016-06-24T21:20:52.367248438Z",
      "google":"1970-01-01T00:00:00Z",
      "meta":"2016-06-24T21:20:45.751852586Z",
      "textrazor":"1970-01-01T00:00:00Z"
    },
    "version":4
  },
  "message":"success",
  "status":200
}

Content Recommendation

/api/content/recommend/user/{fieldname}/{fieldval}?{limit,ql,contentsegment,shuffle,visited,topics,rollups,from,to,path,domain,rank,wait}

Find suggested content for a user based on their known affinities

Parameters

fieldname
string (required)
Ex: email

field to use to identify a user

fieldval
string (required)
Ex: [email protected]

unique identity

contentsegment
string (required)
Ex: blog_content

id, or slug of a content-table segment to filter content

limit
int (optional)
Ex: 20

limit on number of documents to suggest/return

shuffle
bool (optional)
Ex: false

whether or not to shuffle the recommendation order

visited
bool (optional)
Ex: false

whether to show recommendations for content the user has already viewed

topics
[]string (optional)
Ex: ["Marketing", "Analytics"]

if supplied, only allow recommendations on content with the specified topics

rollups
[]string (optional)
Ex: ["Data Topics", "Strategies"]

if supplied, only allow recommendations on content that have relevance to a topic rollup

from
string (optional)
Ex: 2016-01-01T00:00:00Z

Start of time range to include in recommendations.

to
string (optional)
Ex: 2016-06-01

End of time range to include in recommendations.

path
string (optional)
Ex: /blog/archive

Path to match for inclusion in recommendations

domain
string (optional)
Ex: blog.facebook.com

Domain to match for inclusion in recommendations

rank
string (optional)
Ex: affinity

Ranking method for order of recommendations. Must be one of “popular”, “recent”, or “affinity”

wait
bool (optional)
Ex: true

Whether or not to wait for more complex user identities to resolve

GET

Content Recommendation

/api/content/recommend/user/{fieldname}/{fieldval}?{limit,ql,contentsegment,shuffle,visited,topics,rollups,from,to,path,domain,rank,wait}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{  
  "data":[  
    {  
      "url":"www.getlytics.com/blog/post/intent_through_content",
      "title":"Understanding User Intent Through Content",
      "description":"Customer Intent comes from understanding the connection between engagement and its contexts. Content modeling and affinity graphs makes interpretation of intent a lot easier. Learn how marketers can benefit from this.",
      "topics":[  
        "customer data platform",
        "data science",
        "content marketing",
        "personalization"
      ],
      "topic_relevances":{  
        "content marketing":1,
        "customer data platform":1,
        "data science":1,
        "personalization":1
      },
      "primary_image":"https://www.getlytics.com/img/blog/posts/intent_through_content/intent_through_content-bg.gif",
      "author":"",
      "created":"2016-06-08T16:11:45.599594093Z",
      "id":"-3435113786560588929",
      "sitename":"Lytics",
      "stream":"default",
      "path":[  
        "blog",
        "blog/post",
        "blog/post/intent_through_content"
      ],
      "aspects":[  
        "article"
      ],
      "language":"unknown",
      "updated":"2016-06-09T12:49:36.870359765Z",
      "fetched":"2016-06-09T12:49:35.812654441Z",
      "meta":[  
        "og:locale/en_us",
        "og:site_name/lytics",
        "og:title/understanding user intent through content",
        "og:type/article",
        "og:url/https://www.getlytics.com/blog/post/intent_through_content"
      ],
      "confidence":0.5916079783099615,
      "visited":true
    }
  ],
  "message":"success",
  "status":200
}

Content Headline Classify

/api/content/doc/headline?{text,segment_id}

How does a given headline compare to your most and least viral headlines

Parameters

text
string (required)
Ex: 10 Reasons You Ought To Read This Article

subject line

segment_id
string (optional)
Ex: 123abc

segment to filter results from

POST

Content Recommendation

/api/content/doc/headline?{text,segment_id}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data":{
        "title":"10 Reasons You Ought To Read This Article",
        "virality":0.9302325581395349,
        "words":[
            {"word":"you", "common":true, "virality": 0 },
            {"word":"to", "common":true, "virality":0 },
            {"word":"reasons", "common":false, "virality":0.6666666666666666}
        ]
    },
    "message":"success",
    "status":200
}

Content Taxonomy

/api/content/taxonomy{?size,budget,iter,raw,rawThreshold,cliques,cliqueThreshold}

Get the relationships among topics. This gives the number of documents (content items) having each topic.

There are two methods for generating topic relationships from the content taxonomy — through Monte Carlo simulation and through direct calculation from its Bayesian network. The generation method defaults to simulation.

Parameters

size
integer (optional)
Ex: 150

number of topics to include in the topic graph, capped at 500 topics

budget
float (optional)
Ex: 1.0

the depth at which any Monte Carlo simulation can traverse through the graph

iter
integer (optional)
Ex: 8

how many Monte Carlo simulations to run for each topic

raw
bool (optional)
Ex: false

whether to generate the topic relationships from its Bayesian network

rawThreshold
float (optional)
Ex: 0.7

when extracting relationships directly from the Bayesian network, the threshold at which to determine ties

cliques
boolean (optional)
Ex: false

whether to return a single graph or an array of disjoint subgraphs

cliqueThreshold
float (optional)
Ex: 0.0

when returning cliques, the threshold at which to determine if two cliques should be connected

GET

Content Taxonomy

/api/content/taxonomy{?size,budget,iter,raw,rawThreshold,cliques,cliqueThreshold}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
  "data": {
    "n": 438,
    "nodes": [
      {
        "name": "marketing technology",
        "doc_count": 2
      },
      {
        "name": "AdWords",
        "doc_count": 1
      },
      {
        "name": "ad retargeting",
        "doc_count": 4
      },
      {
        "name": "analytics",
        "doc_count": 2
      },
      {
        "name": "customer service",
        "doc_count": 1
      },
      {
        "name": "data visualization",
        "doc_count": 2
      },
      {
        "name": "data-first marketing",
        "doc_count": 157
      },
      {
        "name": "personalization",
        "doc_count": 15
      }
    ],
    "links": [
      {
        "source": 66,
        "target": 54,
        "value": 0.375
      },
      {
        "source": 66,
        "target": 75,
        "value": 0.25
      },
      {
        "source": 66,
        "target": 37,
        "value": 0.25
      }
    ]
  },
  "message": "success",
  "status": 200
}

Content Corpus

/api/content/corpus?{url,text,topics}

Update the corpus, or collection of content documents, that powers the Content Affinity Engine.

In most cases, content is either scraped when the Content Affinity Engine is turned on, or observed as users engage with content from their activity streams. The corpus API provides methods for modifying the corpus — adding custom topics to documents, adding documents for URLs that are private, unlisted, or otherwise irretrievable, etc.

Parameters

url
string (optional)
Ex: http://www.somedomain.com/blog/great-post

the URL for identifying and retrieving content

text
string (optional)
Ex: here is some body text

the body of the content to be added

topics
[]string (optional)
Ex: Data,Hockey

custom topics to append to the document, in addition to topics that will be added with enrichment

POST

Content Corpus

/api/content/corpus?{url,text,topics}

Add a document to the content table corpus. Either a URL or text must be provided. This allows entry of documents/content that get added to content table, content-graph and available for usage in recommendation, as well as content-explorer.

  • id if you upload the document more than once, ensure it has common id so only one document is created

  • Custom Attributes Any attributes you upload that are not reserved, will be added to content document.


# Upsert a document into content table and classify it
curl -s -XPOST "https://api.lytics.io/api/content/corpus" \
  -H "Authorization: $LIOKEY" \
  -H "Content-Type: application/json" \
  -d '{
    "id":"doc_id_123",
    "text":"big-long-body-of text for classification",
    "custom_attribute_name":"custom attribute"
}' | jq '.'


# upsert a url into db
#  url is used as document id
curl -s -XPOST "https://api.lytics.io/api/content/corpus" \
  -H "Authorization: $LIOKEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url":"http://www.validdomain/validurl.html",
    "custom_attribute_name":"custom attribute"
}' | jq '.'

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": 200,
    "message": "success",
    "data": {
      "body": [
        "here is some body text"
      ],
      "source": [
        "upload"
      ],
      "updated": [
        "2016-09-13 21:59:38.551972802 -0400 EDT"
      ],
      "url": [
        "http://www.google.com/"
      ],
      "version": [
        "4"
      ]
    }  
}

System Events

Get internal Lytics system events. These events are generally related to internal changes to state of an account. Common changes are CRUD Operations (Create, Update, Delete) of Account, Admin User, etc.


System Events

/api/events{?id,type,verb,limit,before,after}

Get list of system events.

Parameters

before
string (optional)
Default: now

Start fetching items with dates older than: “now-3d” (now-2h, now-1w, “2016-03-03”). Date math, or absolute date.

after
string (optional)
Default: now

Start fetching items with dates after: “now-3d” (now-2h, now-1w, “2016-03-03”). Date math, or absolute date.

limit
integer (optional)
Default: 20

Should segments this user is member of be included?

verb
string (optional)
Default: ``

Verb (created, updated, deleted, status, clustered)

type
string (optional)
Default: ``

Type of Object (entity, segment, work, account, user)

id
string (optional)
Default: ``

Id of Object (entity, segment, work, account, user)

GET

System Events Query

/api/events{?id,type,verb,limit,before,after}

Get list of system events.

# get the dashboard news items
curl -s -H "Authorization: $LIOKEY" \
  -XGET "https://api.lytics.io/api/event?query=dashboard_news" | jq '.'

# get list of all segments created in last 2 weeks
curl -s -H "Authorization: $LIOKEY" \
  -XGET "https://api.lytics.io/api/event?type=segment&start=now-2w&limit=100&verb=created" | jq '.'


# get list of all events performed by user X

# 1) first find a user id
curl -s -H "Authorization: $LIOKEY" \
  -XGET "https:[email protected]" | jq '.'

# 2) get the id for that user
curl -s -H "Authorization: $LIOKEY" \
  -XGET "https://api.lytics.io/api/event?type=user&id=USER_ID_FROM_PREVIOUS" | jq '.'


Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": [
        {
            "id": "13564999017e8b0e9994ff16411fd19b",
            "ts": "1465584982676",
            "priority": 2,
            "subject_type": "topic-document",
            "subject_id": "-1894399319812759219",
            "verb": "created",
            "account_id": "1234abcdefgh"
        },
        {
            "id": "c57172d7d22e1c7ac4c80ce713e6e7cb",
            "ts": "1465844458666",
            "priority": 2,
            "subject_type": "entity",
            "subject_id": "[email protected]",
            "verb": "clustered",
            "related": [
                {
                    "type": "segment",
                    "id": "791734b084019d99c82a475264464304"
                }
            ],
            "account_id": "1234abcdefgh",
            "context": {
                "in": "2016-06-13T18:58:29Z"
            }
        }
    ],
    "message": "success",
    "status": 200
}

System Event

/api/events/:id
GET

System Event Fetch

/api/events/:id

Get Single System Event

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "id": "c57172d7d22e1c7ac4c80ce713e6e7cb",
        "ts": "1465844458666",
        "priority": 2,
        "subject_type": "entity",
        "subject_id": "[email protected]",
        "verb": "clustered",
        "related": [
            {
                "type": "segment",
                "id": "791734b084019d99c82a475264464304"
            }
        ],
        "account_id": "1234abcdefgh",
        "context": {
            "in": "2016-06-13T18:58:29Z"
        }
    },
    "message": "success",
    "status": 200
}

Catalog

Catalog exposes info about the schema of the data in Lytics.

  • Entities are Object Types, the primary one being a User although most accounts also have Content and Campaign as well.
    Custom object types may be created.

  • Streams Streams are data sources collected from api or Integrations (aka Work).

  • Schema Per entity type is list of field-types.
    Think of it as describing columns on the user table in a database.


Streams

/api/schema/_streams

List of Streams (raw data sources) and which fields they contain.

This shows information like name, data-type (introspected), first-seen, cardinality (number of unique values).

This data is created using probablistic data-structures and also is sampled after 40m events seen per stream. So it may not be exactly correct for cardinalty or unique values after 40 million.


# Example of getting the streams and field's for an account and saving to file
curl -s "$LIOAPI/api/schema/_streams?access_token=$LIOKEY" > /tmp/streams.json

# create a list of stream field names for a stream
jq '.data[] | select(.stream == "cm_activity") | [.fields[].name] | join(",") ' < /tmp/streams.json

GET

Streams

/api/schema/_streams

Get list of streams, field info they contain.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data":[
        {
            "stream":"events",
            "hidden": false,
            "ct": 37, 
            "curct": 0, 
            "last_msg_ts": "1401216931893", 
            "last_update_ts": "1421167622683", 
            "metrics":[
                    {"ct":56,"ts":1390809000},
                    {"ct":44,"ts":1390808000}
            ],
            "fields":[
                {
                    "avg": 4, 
                    "card": 22, 
                    "is_array": true, 
                    "name": "utm_medium", 
                    "sample": [
                        "cpc",
                        "adwords",
                        "blog",
                        "pricing-page",
                        "social",
                        "integrations-page",
                        "email",
                        "Integrations-page",
                        "docs",
                        "search",
                        "CPC",
                        "twitter",
                        "social_organic",
                        "card",
                        "social_paid",
                        "sponsored",
                        "mailchimp",
                        "facebook",
                        "infographic",
                        "lytics-reporting",
                        "retargeting",
                        "referral"
                    ], 
                    "smart": {
                        "coverage": 0, 
                        "email": 0, 
                        "noun": 13, 
                        "time": 0, 
                        "uid": 0, 
                        "value": 0, 
                        "verb": 13,
                        "ip": 0,
                        "phone": 0, 
                        "url": 0,
                        "analyses": 315
                    }, 
                    "sumlen": 146221, 
                    "type": "string", 
                    "usage": 32484,
                    "first_seen": "2016-06-08T16:11:06Z",
                    "last_seen": "2016-06-24T18:00:40.871025224Z",
                    "hidden": false,
                    "column": [
                        {"table": "campaign", "as": "utm_mediums"},
                        {"table": "user", "as": "utm_mediums"}
                    ]
                }
            ]
        }
    ]
}

Stream

/api/schema/_streams/{stream}

Get Single Stream by stream name.

Parameters

stream
string (optional)
Ex: web_data

Name of the stream

PUT

Stream Hide

/api/schema/_streams/{stream}

The PUT endpoint allows Stream to be hidden/un-hidden. A hidden stream is not visible from the Application Data > Streams Web UI.

CAVEAT’s

  • This does Not delete the data, only hides it from ui.

  • If queries (LQL) still map this data into profiles it will still be added.

# Hide a temporary stream
curl -XDELETE 'https://api.lytics.io/api/schema/_streams/my_temp_upload' \
    -H 'Content-type: application/json' \
    -H "Authorization: $LIOKEY" \
    -d '{
        "hidden": true
     }' | jq '.'

Parameters

stream
string (required)
Ex: my_temp_upload

Name of the stream To Delete

hidden
boolean (required)
Ex: true

hide stream from Web UI

Requests

Copy All
Header
Authorization: QWxhZGRpbjpvcGVuIHNlc2FtZQ==

Response 204


Schema

/api/schema/{table}

Field level info for entity/table (usually a user).
but you can try content as well.

# Get the schema for the user table
curl -s -XGET 'https://api.lytics.io/api/schema/user' \
    -H 'Content-type: application/json' \
    -H "Authorization: $LIOKEY"  | jq '.'

Parameters

table
string (required)
Ex: user

Name

GET

Schema Fields

/api/schema/{table}

Get Field information about a NodeType(user). Eventually, we will extend beyond user.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "fields": [
              {
                "as": "created_ts",
                "is_by": false,
                "type": "date",
                "shortdesc": "Created Date",
                "longdesc": "First time usage for this user",
                "froms": [
                  "default"
                ],
                "identities": [
                  "min"
                ]
              },
              {
                "as": "content",
                "is_by": false,
                "type": "string",
                "shortdesc": "Site Content",
                "longdesc": "Specific piece of content viewed",
                "froms": [
                  "default"
                ],
                "identities": [
                  "content"
                ]
              },
              {
                "as": "location",
                "is_by": false,
                "type": "string",
                "shortdesc": "Location",
                "longdesc": "",
                "froms": [
                  "linkedin_data"
                ],
                "identities": [
                  "location"
                ]
              }
        ]
     }
}

Stream Metrics

/api/schema/_streams/metrics

List of metrics (hourly event counts) per stream


curl -s -H "Authorization: $LIOKEY" \
  "https://api.lytics.io/api/schema/_streams/metrics?stream=default" | jq '.'

GET

Stream Metrics

/api/schema/_streams/metrics

Get list of metrics for count of events per stream.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
  "data": [
    {
      "value": 23,
      "ts": "1462930728201"
    },
    {
      "value": 13,
      "ts": "1462934328219"
    },
    {
      "value": 23,
      "ts": "1462937928202"
    }
  ],
  "message": "success",
  "status": 200
}

FieldInfo

/api/schema/{table}/fieldinfo{?limit,fields}

Estimates of field level info for table user segment fields.

Parameters

table
string (required)
Ex: user

Table name (generally “user”)

limit
integer (optional)
Ex: 20

number of unique values to return per field

fields
[]string (optional)
Ex: name,email,id

list, or comma separated list of string fields to include

GET

FieldInfo For Table

/api/schema/{table}/fieldinfo{?limit,fields}

Get Field level info about a NodeType(user). Contains the possible values contained in each field.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "fields":[
              {
                "field": "region",
                "terms_counts": {
                  "02": 704,
                  "07": 857,
                  "AZ": 671,
                  "CA": 18064,
                  "CO": 654,
                  "FL": 1537,
                  "GA": 822,
                  "H9": 758,
                  "IL": 1254,
                  "KS": 1104,
                  "MA": 1301,
                  "NC": 626,
                  "NJ": 892,
                  "NY": 4035,
                  "ON": 1097,
                  "OR": 12158,
                  "PA": 1040,
                  "TX": 2726,
                  "VA": 1894,
                  "WA": 1948
                },
                "more_terms": true,
                "ents_present": 70489,
                "ents_absent": 65796,
                "high_card": false,
                "approx_cardinality": 331
              },
              {
                "field": "utm_mediums",
                "terms_counts": {
                  "Integrations-page": 1102,
                  "adwords": 4434,
                  "banner": 1496,
                  "blog": 3413,
                  "card": 10,
                  "cpc": 2923,
                  "email": 6051,
                  "ppc": 220,
                  "search": 8762,
                  "social": 12513,
                  "parnter": 2314
                },
                "more_terms": true,
                "ents_present": 29595,
                "ents_absent": 116690,
                "high_card": false,
                "approx_cardinality": 27
              },
              {
                "field": "zd_yymm",
                "terms_counts": {
                  "1409": 5,
                  "1410": 12,
                  "1411": 4,
                  "1412": 4,
                  "1501": 3,
                  "1502": 10,
                  "1503": 26,
                  "1504": 20,
                  "1505": 3
                },
                "more_terms": false,
                "ents_present": 72,
                "ents_absent": 136213,
                "high_card": false,
                "approx_cardinality": 9
              },
              {
                "field": "openct",
                "terms_counts": {
                  "1": 75,
                  "10": 18,
                  "108": 3,
                  "11": 7,
                  "12": 5,
                  "152": 3,
                  "19": 8,
                  "2": 42,
                  "20": 15,
                  "24": 3,
                  "26": 10,
                  "3": 11,
                  "32": 3,
                  "35": 3,
                  "4": 25,
                  "5": 16,
                  "51": 15,
                  "6": 17,
                  "8": 5,
                  "9": 11
                },
                "more_terms": true,
                "ents_present": 328,
                "ents_absent": 135957,
                "high_card": false,
                "approx_cardinality": 98,
                "stats": {
                  "mean": 15.621951219512194,
                  "sd": 27.978602557504388,
                  "min": 1,
                  "max": 264,
                  "n": 328
                }
              },
              {
                "field": "mc_lastopen_ts",
                "terms_counts": {},
                "more_terms": false,
                "histograms": [
                  {
                    "data": {
                      "1446336000000": 2,
                      "1446595200000": 2,
                      "1447891200000": 333,
                      "1448150400000": 348,
                      "1448409600000": 9,
                      "1448668800000": 8
                    },
                    "start": "1446401936511",
                    "end": "1448993936511",
                    "interval": "3d"
                  },
                  {
                    "data": {
                      "1442188800000": 35,
                      "1442793600000": 3,
                      "1445212800000": 71,
                      "1445817600000": 2,
                      "1446422400000": 4,
                      "1447632000000": 340,
                      "1448236800000": 351,
                      "1448841600000": 7
                    },
                    "start": "1441131536511",
                    "end": "1448993936511",
                    "interval": "week"
                  },
                  {
                    "data": {
                      "1417392000000": 3,
                      "1420070400000": 2,
                      "1425168000000": 2,
                      "1427846400000": 43,
                      "1430438400000": 5,
                      "1433116800000": 24,
                      "1435708800000": 41,
                      "1438387200000": 48,
                      "1441065600000": 38,
                      "1443657600000": 73,
                      "1446336000000": 702
                    },
                    "start": "1417457936511",
                    "end": "1448993936511",
                    "interval": "month"
                  }
                ],
                "ents_present": 1377,
                "ents_absent": 134908,
                "high_card": false,
                "approx_cardinality": 1212
              }

        ]
     }
}

FieldSuggest

/api/schema/{table}/fieldsuggest/{field}

Suggest Values for a field for auto-complete in webui

Parameters

table
string (required)
Ex: user

Table name (generally user)

field
string (required)
Ex: email

field name to do lookup on.

GET

FieldSuggest

/api/schema/{table}/fieldsuggest/{field}

Suggest Values for a field for auto-complete in webui

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "fields": :[
          {
            "name": "_cc",
            "sample": ["US"],
            "type": "string",
            "usage": 4
          }
        ]
     }
}

Account

Account related resources of the Lytics API.

The Account resource describes an account. An account can be a Master account, which means that it is a billing account. Master accounts have children accounts which act more as separate projects, for same billing account. Each child account can be used for Dev, Testing, etc., or for separate siloes of data for sub-divisions, etc..

The Account resource has the following attributes:

field DataType Description
id string unique id
created Date Created date
updated Date Last Updated Date
domain string customer-assigned unique id (auto assigned to domain name). This must be unique across lytics accounts
email string primary email contact for account owner
parent_id string parent account if this account is a child-account of master billing account
apikey string WARNING-will be deprecated 2017 replaced with tokens api endpoint. api key for account, admin access
dataapikey string WARNING-will be deprecated 2017 api key for account, data read only
name string account name
whitelist_fields []string Whitelisted fields to be used in public entity API
whitelist_domains []string Whitelisted domains that are allowed to use public entity API

Whitelist Fields For the attribute _whitelistfields on account entity model, these are list of fields that will be returned by default on public entity api’s.


Account

/api/account/{id}

Account Api.

Parameters

id
string (required)
Ex: 1234abcd

id of account.

GET

Retrieve a Single Account

/api/account/{id}

Get single account by id.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "apikey": "ceHNqmr0ELpNTjRhVg7FFpzM",
    "created": "2014-10-30T21:01:06.493Z",
    "updated": "2014-10-30T21:01:06.493Z", 
    "dataapikey": "qQsOcWCe-QipNGC2n5NRVVH7",
    "domain": "example.lytics.io",
    "email": "[email protected]",
    "id": "CNakHAobwEeLuUyrKc5AQTif",
    "name": "Lytics Test",
    "parent_id": "CNakHAobwEeLuUyrKc5AQTif",
    "timezone": "America/Los_Angeles",
    "whitelist_domains": [
        "lytics.io",
        "getlytics.com"
    ], 
    "whitelist_fields": [
        "first_name",
        "last_name",
        "sal",
        "zip",
        "country",
        "city",
        "state"
    ]
}
PUT

Account Update

/api/account/{id}

Post update to account. See list of fields above.


curl -XPUT "https://api.lytics.io/api/account/$ACCOUNTID" \
   -H 'Content-type: application/json' \
   -H 'Authorization: $LIOKEY' \
   -d '{
    "whitelist_domains" : ["yourdomain.com","yourotherdomain.com"]
    , "whitelist_fields" :["first_name","last_name","sal","zip","country","city","state"]
}'

Requests

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "whitelist_domains": ["yourdomain.com","yourotherdomain.com"]
}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "apikey": "ceHNqmr0ELpNTjRhVg7FFpzM",
    "created": "2014-10-30T21:01:06.493Z",
    "updated": "2014-10-30T21:01:06.493Z", 
    "dataapikey": "qQsOcWCe-QipNGC2n5NRVVH7",
    "domain": "example.lytics.io",
    "email": "[email protected]",
    "id": "CNakHAobwEeLuUyrKc5AQTif",
    "name": "Lytics Test",
    "parent_id": "CNakHAobwEeLuUyrKc5AQTif",
    "timezone": "America/Los_Angeles",
    "whitelist_domains": [
        "lytics.io",
        "getlytics.com"
    ], 
    "whitelist_fields": [
        "first_name",
        "last_name",
        "sal",
        "zip",
        "country",
        "city",
        "state"
    ]
}
POST

Account Create

/api/account/{id}

To create a child Account of a parent account, just post:

Requests

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "name": "String content",
    "domain":"yourdomain.com"
}

Response 201

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "apikey": "ceHNqmr0ELpNTjRhVg7FFpzM",
    "created": "2014-10-30T21:01:06.493Z",
    "updated": "2014-10-30T21:01:06.493Z", 
    "dataapikey": "qQsOcWCe-QipNGC2n5NRVVH7",
    "domain": "example.lytics.io",
    "email": "[email protected]",
    "id": "CNakHAobwEeLuUyrKc5AQTif",
    "name": "Lytics Test",
    "parent_id": "CNakHAobwEeLuUyrKc5AQTif",
    "timezone": "America/Los_Angeles",
    "whitelist_domains": [
        "lytics.io",
        "getlytics.com"
    ], 
    "whitelist_fields": [
        "first_name",
        "last_name",
        "sal",
        "zip",
        "country",
        "city",
        "state"
    ]
}

Accounts List

/api/account

Collection of all Accounts.

GET

Accounts List

/api/account

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": "success",
    "data": [
        {
            "apikey": "ceHNqmr0ELpNTjRhVg7FFpzM",
            "created": "2014-10-30T21:01:06.493Z",
            "dataapikey": "qQsOcWCe-QipNGC2n5NRVVH7",
            "domain": "example.lytics.io",
            "email": "[email protected]",
            "id": "CNakHAobwEeLuUyrKc5AQTif",
            "name": "Lytics Test",
            "parent_id": "CNakHAobwEeLuUyrKc5AQTif",
            "features":{
                "segment-sizes":true
            },
            "whitelist_domains": [
                "lytics.io", 
                "getlytics.com"
            ], 
            "whitelist_fields": [
                "first_name",
                "last_name",
                "sal",
                "zip",
                "country",
                "city",
                "state"
            ]
        }
    ]
}

User

Info about Users of Lytics that have access to your account.

  • Lytics users are unique across accounts

  • They get invited to have access to your account


User List

/api/user

Get a list of all users for an account.

GET

User List

/api/user

Get list of Admin Users

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": [
        {
            "account_id": "4f7b175bdba058fc092357a6", 
            "created": "2014-08-04T21:18:20.124Z",
            "updated": "2014-08-04T21:18:20.124Z",
            "email": "[email protected]", 
            "id": "120642b03a2d54551e637f56cbf61b73", 
            "name": "Bill Jenkins", 
            "accounts":[
                {
                    "aid":122,
                    "account_id":"4f7b175bdba058fc092357a6",
                    "granted_by":"500418690088145",
                    "roles": ["api", "segments", "admin"],
                    "created": "2014-08-04T21:18:20.124Z",
                    "updated": "2014-08-04T21:18:20.124Z"
                }
            ]
        }
    ]
}

User

/api/user/{userid}

Get Single User

GET

User Fetch

/api/user/{userid}

Get A single Admin User

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "account_id": "4f7b175bdba058fc092357a6", 
        "created": "2014-08-04T21:18:20.124Z",
        "updated": "2014-08-04T21:18:20.124Z",
        "email": "[email protected]", 
        "id": "120642b03a2d54551e637f56cbf61b73", 
        "name": "Bill Jenkins", 
        "accounts":[
            {
                "aid":122,
                "account_id":"4f7b175bdba058fc092357a6",
                "granted_by":"500418690088145",
                "roles": ["api", "segments", "admin"],
                "created": "2014-08-04T21:18:20.124Z",
                "updated": "2014-08-04T21:18:20.124Z"
            }
        ]
    }
}
POST

User Create

/api/user/{userid}

Create or Invite A Lytics Admin User this will either create a new user if that email is new, or invite user to have access to current account.

# Create/Invite new user for current LIOKEY account
curl -v -XPOST https://api.lytics.io/api/user?access_token=$TOKEN  \
  -H 'Content-type: application/json' \
  -d '{
    "email" : "[email protected]"
    , "name":"aaron"
    , "roles": ["api","admin","segments"]
}'

Parameters

suppress
bool (optional)
Default: true

default = false, supress email invite

Requests

Copy All
Header
Content-Type: application/json
Copy All
Header
Authorization: QWxhZGRpbjpvcGVuIHNlc2FtZQ==
Copy All
Body
{
    "name": "Bob",
    "email": "[email protected]"
}

Response 201

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "account_id": "4f7b175bdba058fc092357a6", 
        "created": "2014-08-04T21:18:20.124Z",
        "updated": "2014-08-04T21:18:20.124Z",
        "email": "[email protected]", 
        "id": "120642b03a2d54551e637f56cbf61b73", 
        "name": "Bill Jenkins", 
        "accounts":[
            {
                "aid":122,
                "account_id":"4f7b175bdba058fc092357a6",
                "granted_by":"500418690088145",
                "roles": ["api", "segments", "admin"],
                "created": "2014-08-04T21:18:20.124Z",
                "updated": "2014-08-04T21:18:20.124Z"
            }
        ]
    }
}
PUT

User Update

/api/user/{userid}

Update An Lytics Admin User

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "account_id": "4f7b175bdba058fc092357a6", 
        "created": "2014-08-04T21:18:20.124Z",
        "updated": "2014-08-04T21:18:20.124Z",
        "email": "[email protected]", 
        "id": "120642b03a2d54551e637f56cbf61b73", 
        "name": "Bill Jenkins", 
        "accounts":[
            {
                "aid":122,
                "account_id":"4f7b175bdba058fc092357a6",
                "granted_by":"500418690088145",
                "roles": ["api", "segments", "admin"],
                "created": "2014-08-04T21:18:20.124Z",
                "updated": "2014-08-04T21:18:20.124Z"
            }
        ]
    }
}

Auth

API for creation or storage of auth tokens. Most of these tokens are created from the Integrations/Imports setup for Lytics integrations.

Additionally, api tokens can be created/managed from this endpoint.

The oauth, config information are write only attributes that cannot be subsequently read from the api, and is stored encrypted.


Auth

/api/auth/{id}{?access_token}

A single Auth object. The Auth resource is the central resource in the Lytics API.

The Auth resource has the following attributes:

field DataType Description
id string unique id, Lytics assigned
account_id string Unique account id, lytics assigned
created Date Created date
updated Date Last Updated Date
provider_id string the id of the provider resource that this auth is used for
user_id string the id of the user resource that originally provided this auth (optional)
user_name string Name of the user that provided auth
config object Json object of name/value pairs that provides auth info Writeonly attribute
oauthtoken object oauth1 Writeonly attribute
oauth2token object oauth2 Writeonly attribute

Parameters

id
string (required)

ID of the Gist in the form of a hash.

access_token
string (optional)

Lytics API access token.

GET

Retrieve a Single Auth

/api/auth/{id}{?access_token}

This API is fairly special, as only the public portions of the auth model are returned, not all fields config, oauth2token, and oathtoken fields are never exposed.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status":"success",
    "data":{
       "account_id": "4f7b525bdba058fc096757a6", 
        "id": "1b53b88a06b418b07d601b6f57b8eb40", 
         "provider_id": "26cae7718c32180a7a0f8e19d6d40a59", 
        "provider_name": "Facebook", 
        "user_id": "4222fd7c8db7ebcb042e0547e3b4e128"
    }
}
POST

Create Auth

/api/auth/{id}{?access_token}

Create an auth, would normally never be used via the api, the web admin Integrations section manages these.

Requests

Copy All
Header
Content-Type: application/json
Copy All
Header
Authorization: QWxhZGRpbjpvcGVuIHNlc2FtZQ
Copy All
Body
{
    "provider_id": "our_provider",
    "config":[
        {"name":"apikey","value":"value_that_will_get_encrypted"}
    ]

}

Response 201

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status":"success",
    "data":{
       "account_id": "4f7b525bdba058fc096757a6", 
        "id": "1b53b88a06b418b07d601b6f57b8eb40", 
         "provider_id": "26cae7718c32180a7a0f8e19d6d40a59", 
        "provider_name": "Facebook", 
        "user_id": "4222fd7c8db7ebcb042e0547e3b4e128"
    }
}
PUT

Update Auth

/api/auth/{id}{?access_token}

Requests

Copy All
Header
Content-Type: application/json
Copy All
Header
Authorization: QWxhZGRpbjpvcGVuIHNlc2FtZQ
Copy All
Body
{
    "provider_id": "our_provider",
    "config":[
        {"name":"apikey","value":"value_that_will_get_encrypted"}
    ]
    "oauth2token": {.....},
    "oauthtoken": {....}

}

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status":"success",
    "data":{
       "account_id": "4f7b525bdba058fc096757a6", 
        "id": "1b53b88a06b418b07d601b6f57b8eb40", 
         "provider_id": "26cae7718c32180a7a0f8e19d6d40a59", 
        "provider_name": "Facebook", 
        "user_id": "4222fd7c8db7ebcb042e0547e3b4e128"
    }
}
DELETE

Remove an Auth

/api/auth/{id}{?access_token}

Deleting an auth is a very special operation. If any work is running (imports, exports), it will remove the auth from those works and stop them.

Requests

Copy All
Header
Authorization: QWxhZGRpbjpvcGVuIHNlc2FtZQ==

Response 204


Auth ApiToken

/api/auth/createtoken

Create Lytics api tokens.

Please note that the response below has the value of the api key created.

This api key will NOT be available via the api outside of this single response so you must save it safely.

Subsequent usage of this auth may be managed through normal Auth api above.

Parameters:

  • expires 100h (hours) this token will only be valid until this many hours have passed

  • name user-provided name for this token

  • scopes comma delimited list of scopes (admin,data)

Scopes

  • admin can manage the account (upload queries, create users)

  • data can upload data, read user-profiles but not manage account.

POST

Auth Create Token

/api/auth/createtoken

Create Lytics api tokens.

Requests

Copy All
Header
Content-Type: application/json
Copy All
Header
Authorization: QWxhZGRpbjpvcGVuIHNlc2FtZQ
Copy All
Body
{
    "expires":"100h",
    "scopes": ["admin","data","api"],
    "name": "my-token-for-serivce-x"
}

Response 201

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status":"success",
    "data":{
        "account_id": "4f7b525bdba058fc096757a6",
        "id": "1b53b88a06b418b07d601b6f57b8eb40",
        "provider_id": "26cae7718c32180a7a0f8e19d6d40a59",
        "name": "my-token-for-service-x",
        "provider_name": "Lytics",
        "config":[
            { "api_key": "api_value" }
        ]
    }
}

Provider

Provider is a 3rd-party Lytics Integration with (Mailchimp, Optimizely, Mixpanel, Facebook etc). It should not generally be needed unless you need provider ids for managing Auths.


Provider List

/api/provider
GET

Provider List

/api/provider

Provider list API:

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": [
      {
        "auth_description": "", 
        "categories": [
            "source", 
            "social", 
            "export"
        ], 
        "description": "", 
        "hidden": false, 
        "id": "26cae7718c32180a7a0f8e19d6d40a59", 
        "name": "Facebook", 
        "namespace": "fb", 
        "no_auth": false, 
        "oauth2": {
            "auth_endpoint": "https://www.facebook.com/dialog/oauth", 
            "auth_params": {
                "response_type": [
                    "code"
                ]
            }, 
            "client_id": "500418690088145", 
            "redirect_uri": "https://api.lytics.io/api/oauth/receive", 
            "scope": "email,read_insights,manage_pages,ads_management", 
            "token_endpoint": "https://graph.facebook.com/oauth/access_token"
        }, 
        "slug": "facebook"
      },
      ....
    ]
    "message": "success", 
    "status": 200
}

Provider

/api/provider/:provider_slug

Get Single Provider

GET

Provider Fetch

/api/provider/:provider_slug

Get single Provider

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "auth_description": "", 
        "categories": [
            "source", 
            "social", 
            "export"
        ], 
        "description": "", 
        "id": "26cae7718c32180a7a0f8e19d6d40a59", 
        "name": "Facebook", 
        "slug": "facebook"
    }
}

Query

Schema management api to add/edit queries and user-fields. for sub-divisions, etc..

Lytics Query Language

The Lytics Query Language is used to define the transformation of uploaded records, and event data into user Profiles. It transforms row-level event data into Document-oriented User info.
This Query langage is similar to the HIVE or SQL query langauges, however departs from these in order to offer more of a Rich Document (json user profile) construction.

Query example

    
# Build a user from web data
SELECT 
    name                       -- Simple field, by default = string
    , age            KIND INT  -- cast field as int
    , last_visit_ts  KIND DATE -- cast as date
    
    -- Showing the aggregate counter function and aliasing name of output column AS
    , count(_ref)         AS ref_ct

    -- Valuect makes a map[string]int count of occurences of a key
    , valuect(`my field`) AS myfield_mapct

    -- showcase every optional syntax element in column
    --     meregeop oldest we don't want to over-write this value, keep oldest
    --     KIND INT   normally we don't have to cast as most functions have a specific type
    , amt AS first_order_amount
          IF  event == "cart checkout"
          SHORTDESC "Amount of First Order" 
          LONGDESC "Amount of First Order"
          KIND INT
          MERGEOP OLDEST  
   
    -- lets keep around the date at which they signed up (mergeop oldest)
    , now()               AS signedup_date                  IF event == "signed up"    KIND DATE  MERGEOP oldest

    -- maps:   map all fields that start with "user." into a fact map
    , match("user.")        AS user_attributes     KIND map[string]string

    -- list of strings
    , set(event) AS all_events


    -- Identified By Columns allow merging across streams
    , email(EmailAddress) AS email
    , _uid 
    , fbuid

FROM 
    default
INTO
    user
BY 
    _uid OR email OR fbuid
WHERE 
    _bot = "f" OR NOT EXISTS _bot
ALIAS 
    web_user;

# validate the query
curl -s -XPOST "https://api.lytics.io/api/query/_validate" \
   -H "Authorization: $LIOKEY" \
   -H "Content-Type: text/plain" \
   --data-binary @/tmp/tmp.lql | jq '.'

# upload the query
curl -s -XPOST "https://api.lytics.io/api/query" \
   -H "Authorization: $LIOKEY" \
   -H "Content-Type: text/plain" \
   --data-binary @your_file.lql | jq '.'

# look at schema it output:
curl -s -H "Authorization: $LIOKEY" \
 -XGET "https://api.lytics.io/api/schema/user" | jq '.'

Standard Syntax


Select = "SELECT" COLUMNS FROM INTO BY [WHERE] ALIAS

# required from, the stream to operate on for this query
FROM = "FROM" Identifier

# Required Identified By field, name of column "AS" from Column
BY = "BY" Identifier ["OR" Identifier]

# Required Alias for giving a query a unqique identifier
ALIAS = "ALIAS" Identifier

# Optional Where Filter, same as SQL where
WHERE = "WHERE" LogicalExpression 

COLUMNS         = COLUMN [, COLUMN]

COLUMN     = Expression ["AS" Identifier] 
     ["IF" LogicalExpression] ["SHORTDESC" String] 
     ["LONGDESC" String] ["KIND" Kind] ["MERGEOP" MergeOp]

LogicalExpression = NOT
           | Comparison
           | EXISTS
           | IN
           | CONTAINS
           | LIKE
           | Function
           | Expression
           | "(" LogicalExpression ")"
           | LogicalExpression OR LogicalExpression
           | LogicalExpression AND LogicalExpression

Expression = 
    Identifier 
    | Function 
    | Literal

Function = Identifier "(" Expression [, Expression] ")"

NOT            = "NOT" LogicalExpression
Comparison     = Identifier ComparisonOp Literal
ComparisonOp   = ">" | ">=" | "<" | "<=" | "==" | "!="
EXISTS         = "EXISTS" Identifier
IN             = Identifier "IN" (Literal, Literal, ...)
CONTAINS       = Identifier "CONTAINS" Literal
LIKE           = Identifier "LIKE" String # uses * for wildcards


Literal = String | Int | Float | Bool | Timestamp

Identifier = [a-zA-Z][a-zA-Z0-9_]+ | "`" + String + "`"

Kind = "int" | "number" | "string" | "date" | "[]string" | 
    "ts[]string" | "map[string]int" | "map[string]number" | "map[string]string*

# MergeOp's are very seldom used and have to be used on the right Kind
#  ie string can use Latest, Oldest (but not min, max)
MergeOp =  "max" | "min" | "latest" | "oldest" | "mapmax"


  • SELECT Select data to be added to user profiles. Including Maps, Counts, and other complex data types.

  • FROM The stream to select from

  • INTO This is USER for all user profiles. (technically you could create other types, such as “account”)

  • WHERE Filters out entire records to not be included/analyzed. Bots, Employees, Test data.

  • BY What field are we going to identify this entity by

  • ALIAS When a selection query has an alias, that is the profile-fragment(table) name to use

Functions

There are a variety of functions for transformation and logic evaluation.

Aggregate Functions

There are a variety of expressions for building document type structures (maps, lists, sets). These are functional expressions but can only be used in Columns.

  • count Count of this key. For instance, count occurences of sessions that have started (ie, visited web site).

  • valuect Count each occurence of specific value of a field. For instance, valuect on the field event will count how many times a visitor has done each event type (such as the number of times event = login).

  • set Create a unique list/array of each value we have seen from this field

  • min,max Minimum or Maxium value (for numerics)

  • sum Sum values (keep track of total video play time, etc)

Logical Functions Local Evaluation, return boolean true/false.

  • all checks for existince of n keys all(key1,key2,key3,...) returns boolean.

  • any accepts a list of values and returns True if any are the contents of a field any(fieldname, value1,value2,value3)

  • exists Check for field (aka key) existence.

    • exists(purchase_total) checks to see if purchase_total is defined for the current message
    • valuect(yymm()) AS visits_by_yymm IF exists(_sesstart) Only fires valuect(yymm()) if _sesstart exists
  • in Determines if a field value is in a set of values.

    • "t" AS is_student IF role_type IN ("student","other")
    • dailyContact AS dailyContact IF dailyContact IN ("student","other")
  • eq Equal to eq(domain,"google.com")

  • ne Not Equal to ne(domain,"google.com")

  • lt Less Than lt(seconds(video_time), 30)

  • le Less Than or Equal to le(seconds(video_time), 30)

  • gt Greater Than gt(seconds(video_time), 30)

  • ge Greater Than or Equal to ge(seconds(video_time), 30)

  • not Not not(exists(domain))

  • or Or or(exists(domain), contains(domain,"google.com")) AS from_google

String Functions

  • geocity City

  • geoorg Org

  • georegion Region (State)

  • geocountry Country

  • join Join together multiple values, coerce them into strings

  • len Length (of array, string)

  • oneof Choose value(s) from the first field that has values (ie, allow merging fields into one user-field)

    • oneof(fielda,fieldb,fieldc)
  • replace - Replace a matching part of a field with an empty string

    • replace(url,"/search/apachesolr_search/") - Removes /search/apachesolr_search/ from URL (in this case, leaving the search term
  • split Breaks a variable into smaller fragments given a specific delimiter

    • split(cc,",") - Splits the variable cc at each comma it contains
  • strip(field) Strips leading and trailing whitespace (spaces, tabs, newline, carriage-return) from string, or arrays of strings.

  • tolower Convert strings to lower case

  • contains Does this value contain this string?

    • IF contains(total_price, "$") - Check to see if total_price has a $ in it
    • IF not(contains(subscriber_key,"-")) AND not(contains(subscriber_key,"@")) check to make sure - or @ is not in it.
  • hasprefix Does this value start with this string?

    • hasprefix(event, "created") - Check to see if event starts with “created”
  • hassuffix Does this value start with this string?

    • hassuffix(subscriber_key, "user") - Check to see if subscriber_key ends with “user”

String Hash & Encoding Functions

  • hash.sip hash.sip(email) Hash the given value using sip hash to integer output.

  • hash.md5 hash.md5(email) Hash the given value using md5

  • hash.sha1 hash.sha1(email) Hash the given value using sha1

  • hash.sha256 hash.sha256(email) Hash the given value using sha256

  • hash.sha512 hash.sha512(email) Hash the given value using sha512

  • encoding.b64encode(field) base64 encode.

  • encoding.b64decode(field) base64 decode.

Cast & Convert

  • toint Converts strings to integers. Useful for converting a string to a number before applying a number-based expression.

    • toint(order_total) - Converts order_total to an int
    • set(toint(split(cc,","))) - Takes the field cc and splits it at commas, and converts the results to integers. Then adds them to a set.
  • tonumber Convert to Number

  • todate Converts strings to dates, see full doc in Date/Time section below.

  • todate(field) Converts to Number(float).

  • tobool(field) Cast to Boolean.

Map & Set/Array Functions

  • filter Filter out Values that match specified list of match filter criteria

    • filter(split("apples,oranges",","),"ora*") => ["apples"]
  • len Length (of array, string)

  • map Type: Map map(key1, todate(date_field))

    • map(key1, todate(date_field)) KIND map[string]time By default the map is generic map, cast to map[string]time with
  • match Type: Map (generic map, use KIND to cast) Match a key, and then keep a map of key/values with the match value removed

    • , match("topic_") AS global KIND map[string]number
  • mapkeys Type: Map input, []string{} output. Given a map, return a list of string of each of the keys.

  • mapvalues Type: Map input, []string{} output. Given a map, return a list of string values of each of the values.

  • mapinvert Type: Map input, MapString output. Given a map, return a map[string]string inverting keys/values.

  • array.index Cherry pick a single item out of an array:

    • array.index(split("apples,oranges,peaches",","),1) => ["oranges"]
  • array.slice Slice an array of items selecting some sub-set of them.

    • array.slice(split("apples,oranges,peaches,pineapple",","),2) => ["peaches","pineapple"]
    • array.slice(split("apples,oranges,peaches,pineapple",","),1,3) => ["oranges","peaches"]

Url/Http & Email Functions

  • email Extract email address from “Bob <[email protected]>” format

  • emailname Extract Bob from “Bob <[email protected]>” or [email protected]

  • emaildomain Extract gmail.com from “Bob <[email protected]>” or [email protected]

  • host Extract host from url

  • path Extract the url path from url (no query string or domain), must be valid url parserable string.

  • qs Extract the querystring parameter from url qs(urlfield, "nameOfParam")

    • qs(url, "mc_eid") - Extracts the MailChimp user ID
    • set(qs(url, "video_id") - Creates a set of video_id
    • qs(tolower(url), "riid") - Converts the complete URL to lowercase before attempting to match
    • email(oneof(email, qs(url, "email"))) - Attempts to get the email address from the URL and from the regular fields, chooses whichever is populated and treats it like an email field
  • urldecode Perform URL decode on a field. urldecode(field)

    • If field contains “my%20value”, urldecode(field) will return “my value”
  • urlminusqs The url minus the querystring portion

  • useragent Extract info from user-agent string. Below examples based on Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11

    • useragent(user_agent, "bot") - Extracts True/False is this a bot?
    • useragent(user_agent, "mobile") - Extracts True/False is this mobile?
    • useragent(user_agent, "mozilla") - Extracts “5.0”
    • useragent(user_agent, "platform") - Extracts “X11”
    • useragent(user_agent, "os") - Extracts “Linux x86_64”
    • useragent(user_agent, "engine") - Extracts “Linux x86_64”
    • useragent(user_agent, "engine_version") - Extracts “AppleWebKit”
    • useragent(user_agent, "browser") - Extracts “Chrome”
    • useragent(user_agent, "browser_version") - Extracts “23.0.1271.97”
  • useragent.map(field) Extract map of all of above.

Date & Time Functions

Our core date parser recognizes about 50 date formats, so in general these will operate on any format. If you are using EU dates, you will need to specify the parser format.

  • dayofweek Type: Integer. 0-6 integer of day of week.

    • Examples: dayofweek() => 4 OR dayofweek(mydatefield)
  • epochms Type: Integer. Unix MS of the date stamp on the current message being processed

  • extract Can be used to extract parts of date and time. Example usage on the strftime site

    • extract(reg_date, "%B") Returns name of month
    • extract(reg_date, "%d") Returns day of month
  • hourofday Type: Integer. Hour of day (in 24 hour utc time). hourofday() ORhourofday(field)`

  • hourofweek 0-167 integer for hour of week

  • mm Type: Integer. 0-11 month (alias for monthofyear) mm() => current month, 6 for june, mm(my_date_field)

  • monthofyear Type: Integer Output the 0-11 month value

  • now Type: Date The current message/event times.

  • seconds Type: Integer. Seconds, extracts things like seconds("00:30") => 30 and seconds("10:30") => 630

  • todate Converts strings to dates.

    • Datemath: todate("now-3m") Date math relative to message timestamp.
    • Parser: todate("02/01/2006") More than 30 formats supported. Date Parser
    • Examples with 2 arguments: todate("02/01/2006","07/04/2014") use golang’s time package formatting
      • todate("02/01/2006","07/04/2014") Reformats the date 07/04/2014 from US formatting to UK formatting, with the resulting output being 04/07/2014
      • todate("02/01/2006",date_field_name) Outputs date_field_name as European format (where 01 is a placeholder for month, 02 is a placeholder for day, and 2006 is a placeholder for year)
  • totimestamp Convert to Integer Unix Seconds

  • yy Type: int Date conversion to YY format, so May 1 2014 is expressed as 14. yy(dob), or yy() for record time stamp

  • yymm The YYMM date format, so May 1 2014 is expressed as 1405. yy(dob), or yy() for record time stamp

  • timebucket Creates a tabulation of timestamps which can be used to segment based on timewindows. See Segments Examples for more information. timebucket(now()) for collect time, or timebucket(todate(field)) to bucket on the value of a field

KINDS (aka Data Types)

Allows explicitly setting data type. Often this os optional as it is inferred from functional expression.

  • int 64 bit signed integer

  • number 64 bit signed Float value

  • bool Boolean

  • date Date-Time

  • string string

  • []string Set of strings

  • ts[]string Time ordered Set of strings (useful for keeping track of order in which they performed set of unique events)

  • map[string]int Map of key/integers

  • map[string]number

  • map[string]string

  • map[string]time

Merge Operations

MERGEOP Allow Merge behavior’s to determine if given new data we want the new field, or keep the previous.

  • , my_date KIND DATE MERGEOP oldest – Holds the first value seen for my_date

  • , old_score KIND INT MERGEOP oldest – Holds the oldest value passed in to the field

  • set(lists) AS lists KIND []string MERGEOP latest – only store latest set (all previous values of set discarded)


Query

/api/query

Query api

GET

Query Fetch

/api/query

Get single query by ALIAS

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": "success",
    "data": {
        "id": "abcdef123",
        "updated": "2014-10-30T21:01:06.493Z",
        "created": "2014-10-30T21:01:06.493Z",
        "text": "SELECT ..."
    }
}
POST

Query Upsert

/api/query

Uses ALIAS name inside parsed query for ID, and either creates/updates a query.


curl -s -XPOST "https://api.lytics.io/api/query" \
   -H "Authorization: $LIOKEY" \
   --data-binary @your_file.lql

Requests

Copy All
Header
Content-Type: text/plain
Copy All
Body
SELECT .....
FROM stream_name
BY field_name
ALIAS my_query_name

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": "success",
    "data": {
        "id": "abcdef123",
        "updated": "2014-10-30T21:01:06.493Z",
        "created": "2014-10-30T21:01:06.493Z",
        "text": "SELECT ..."
    }
}
GET

Queries List

/api/query

List of all queries

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": "success",
    "data": {
        "id": "abcdef123",
        "updated": "2014-10-30T21:01:06.493Z",
        "created": "2014-10-30T21:01:06.493Z",
        "text": "SELECT ..."
    }
}
DELETE

Query Delete

/api/query

Delete A query by alias.

Response 204

POST

Query Validation

/api/query

Upload a query for syntax validation only.

Optionally, a segments=true parameter can be passed that will validate this query both for syntax, as well as checking that it doesn’t invalidate existing segments. If a segment uses a field that this query is about to remove/alter such that the segment is no longer valid, this will warn.


# validate query syntax
curl -s -XPOST "https://api.lytics.io/api/query/_validate" \
   -H "Authorization: $LIOKEY" \
   -H "Content-Type: text/plain" \
   --data-binary @your_file.lql

# validate query syntax AND segments still valid
curl -s -XPOST "https://api.lytics.io/api/query/_validate?segments=true" \
   -H "Authorization: $LIOKEY" \
   -H "Content-Type: text/plain" \
   --data-binary @your_file.lql

Requests

Copy All
Header
Content-Type: text/plain
Copy All
Body
SELECT .....
FROM stream_name
BY field_name
ALIAS my_query_name

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "status": "success",
    "data": {
        "id": "abcdef123",
        "updated": "2014-10-30T21:01:06.493Z",
        "created": "2014-10-30T21:01:06.493Z",
        "text": "SELECT ..."
    }
}
POST

Query Test Evaluation

/api/query

Upload a query AND data to see how it will be interpreted.


# add any name/value paris to query string param for data input
# then upload query and get evaluation response
curl -s -XPOST "https://api.lytics.io/api/query/_test?name=value" \
   -H "Authorization: $LIOKEY" \
   -H "Content-Type: text/plain" \
   --data-binary @your_file.lql

Requests

Copy All
Header
Content-Type: text/plain
Copy All
Body
SELECT .....
FROM stream_name
BY field_name
ALIAS my_query_name

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "segments": [
            "email_us_based",
            "web_users",
            "all"
        ]
    },
    "message": "success",
    "meta": {
        "by_fields": [
            "user_id",
            "email",
            "fbuid",
            "_uids"
        ],
        "format": "json",
        "name": "user"
    },
    "status": 200
}

SegmentML

SegmentML provides a framework for building custom machine learning models directly in Lytics. Lytics SegmentML models are self-training, continuously-updating and real-time.

SegmentML models are built by identifying:

  1. A segment of users, called the Target Segment who exhibit behavior for prediction,

  2. A segment of users, called the Source Segment to be candidates for model evaluation, or scoring.

Models are built with a variety of pre-selected candidate features, which include behavioral scores and content affinities, and can additionally support any custom field available in Lytics user profiles.


SegmentML

/api/segmentml/{id}
GET

SegmentML Model Fetch

/api/segmentml/{id}

Get a SegmentML model.

Parameters

id
string (required)
Ex: source::target

ID of the SegmentML model to retrieve, of the form SOURCE_SLUG::TARGET_SLUG.

Response 200

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "all::active_prospects": {
            "fieldtypes": {
                "aspect_aspect_mobile_android": "numeric", 
                "aspect_aspect_mobile_ios": "numeric", 
                "aspect_aspect_mobile_opened": "numeric", 
                "aspect_aspect_web_multisession": "numeric", 
                "aspect_aspect_web_singlepage": "numeric", 
                "aspect_aspect_web_visit_mobileweb": "numeric", 
                "aspect_aspect_web_visit_web": "numeric", 
                "lytics_score_frequency": "numeric", 
                "lytics_score_intensity": "numeric", 
                "lytics_score_momentum": "numeric", 
                "lytics_score_propensity": "numeric", 
                "lytics_score_quantity": "numeric", 
                "lytics_score_recency": "numeric", 
                "rob_masters_20_outreach": "numeric"
            }, 
            "importance": {
                "aspect_aspect_mobile_android": 0, 
                "aspect_aspect_mobile_ios": 0, 
                "aspect_aspect_mobile_opened": 0, 
                "aspect_aspect_web_multisession": 0.16233333333333333, 
                "aspect_aspect_web_singlepage": 0, 
                "aspect_aspect_web_visit_mobileweb": 0, 
                "aspect_aspect_web_visit_web": 0, 
                "lytics_score_frequency": 0.5850666666666667, 
                "lytics_score_intensity": 0.4132333333333339, 
                "lytics_score_momentum": 0.42413333333333275, 
                "lytics_score_propensity": 0, 
                "lytics_score_quantity": 0.36269999999999963, 
                "lytics_score_recency": 0.10316666666666663, 
                "rob_masters_20_outreach": 0
            }, 
            "summary": {
                "Conf": {
                    "FalseNegative": 3, 
                    "FalsePositive": 1, 
                    "TrueNegative": 2, 
                    "TruePositive": 2
                }, 
                "Mse": 0.27024152611111113, 
                "Rsq": -0.008901697481481508
            }
        }
    }
}
POST

SegmentML Create

/api/segmentml/{id}

Create a new SegmentML model.

In order to create models with this endpoint, the account must have the use_segmentml account setting. This can be enabled by contacting Lytics Customer Success.

Model configuration can be specified either through sending the options as a flat JSON object POST body or through URL parameters on the request.

# Curl example of creating a SegmentML
curl -s -J -XPOST "https://api.lytics.io/api/segmentml" -H "Authorization: $LIOKEY" -d '
{
    "source": "50f4c7c6e327a73ae8000004",
    "target": "d5975ab901f7f660845c95ee",
    "use_scores": true
}
'

Requests

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "save_model": true,
    "continuous": false,
    "source": "source_segment_id",
    "target": "target_segment_id",
    "use_scores": true,
    "collections": ["Web", "Mobile", "Behavioral"],
    "additional_fields": ["Country", "State", "State::Oregon"]
}

Response 201

Copy All
Header
Content-Type: application/json
Copy All
Body
{
    "data": {
        "all::active_prospects": {
            "fieldtypes": {
                "aspect_aspect_mobile_android": "numeric", 
                "aspect_aspect_mobile_ios": "numeric", 
                "aspect_aspect_mobile_opened": "numeric", 
                "aspect_aspect_web_multisession": "numeric", 
                "aspect_aspect_web_singlepage": "numeric", 
                "aspect_aspect_web_visit_mobileweb": "numeric", 
                "aspect_aspect_web_visit_web": "numeric", 
                "lytics_score_frequency": "numeric", 
                "lytics_score_intensity": "numeric", 
                "lytics_score_momentum": "numeric", 
                "lytics_score_propensity": "numeric", 
                "lytics_score_quantity": "numeric", 
                "lytics_score_recency": "numeric", 
                "rob_masters_20_outreach": "numeric"
            }, 
            "importance": {
                "aspect_aspect_mobile_android": 0, 
                "aspect_aspect_mobile_ios": 0, 
                "aspect_aspect_mobile_opened": 0, 
                "aspect_aspect_web_multisession": 0.16233333333333333, 
                "aspect_aspect_web_singlepage": 0, 
                "aspect_aspect_web_visit_mobileweb": 0, 
                "aspect_aspect_web_visit_web": 0, 
                "lytics_score_frequency": 0.5850666666666667, 
                "lytics_score_intensity": 0.4132333333333339, 
                "lytics_score_momentum": 0.42413333333333275, 
                "lytics_score_propensity": 0, 
                "lytics_score_quantity": 0.36269999999999963, 
                "lytics_score_recency": 0.10316666666666663, 
                "rob_masters_20_outreach": 0
            }, 
            "summary": {
                "Conf": {
                    "FalseNegative": 3, 
                    "FalsePositive": 1, 
                    "TrueNegative": 2, 
                    "TruePositive": 2
                }, 
                "Mse": 0.27024152611111113, 
                "Rsq": -0.008901697481481508
            }
        }
    }
}
DELETE

SegmentML Delete

/api/segmentml/{id}

Delete a SegmentML model.

Parameters

id
string (required)
Ex: source::target

ID of the SegmentML model to delete, of the form SOURCE_SLUG::TARGET_SLUG.

Response 204