Skip to content
WebARKitWebARKitWebARKit Docs

nft-tracker

nft-tracker (@webarkit/nft-tracker) tracks a flat image in a camera feed. It lives in the webarkit monorepo and is written above a computer-vision contract (CvBackend): the caller injects the backend, so it runs on any implementation of it. The reference one is jsfeatNext.

It has three parts:

  • the .wnft format: an in-memory description of a trained target and a codec (decode and encode) for the files that store it
  • compile-target: a command-line script that turns an image into a .wnft
  • NftTracker: a tracker with three states, LOST, DETECT and TRACK

Try it

Section titled “Try it”
Terminal window
git clone https://github.com/webarkit/webarkit.git
cd webarkit
npm install   # links the workspace packages together
npm run build

Compile an image into a target, with its size in millimetres:

Terminal window
node packages/nft-tracker/bin/compile-target.mjs examples/images/pinball.jpg \
    -o examples/targets/pinball.wnft --physical-size 210x262.5

This script is a development tool of the repository, not part of the package.

The tracker

Section titled “The tracker”
import { NftTracker } from '@webarkit/nft-tracker';


const tracker = new NftTracker(cv, target, K); // cv: the CvBackend, K: camera intrinsics
const result = tracker.process(frame, timestampMs); // frame: a GrayImage
console.log(result.state, result.quality);
if (result.ok) {
  // result.H maps the target's pixels into the frame (3×3, row-major)
}

process does not throw on a frame it cannot use: check result.ok. Every result has a state and a quality between 0 and 1:

state Meaning
DETECT The pose comes from the detection pipeline: detect, describe, match, homography
TRACK The pose comes from tracking the target’s patches, without running detection
LOST No pose, and reason says why

A frame without a lock runs detection; once it succeeds, the next frames track the target’s patches and detect again only if the lock is lost. A target without patches is detected on every frame.

Detection is the expensive step (about 79 ms on the reference device, against a 33 ms frame budget). The tracker owns no worker: run prepareDetection and detectTarget in a worker of your own, and hand the result back to process with the externalDetection option.

See also

Section titled “See also”