---
title: Advanced Subgraph Features
---

## Overview

Add and implement advanced Subgraph features to enhance your Subgraph's build.

Starting from `specVersion` `0.0.4`, Subgraph features must be explicitly declared in the `features` section at the top level of the manifest file, using their `camelCase` name, as listed in the table below:

| Feature                                              | Name             |
| ---------------------------------------------------- | ---------------- |
| [Non-fatal errors](#non-fatal-errors)                | `nonFatalErrors` |
| [Full-text Search](#defining-fulltext-search-fields) | `fullTextSearch` |
| [Grafting](#grafting-onto-existing-subgraphs)        | `grafting`       |

For instance, if a Subgraph uses the **Full-Text Search** and the **Non-fatal Errors** features, the `features` field in the manifest should be:

```yaml
specVersion: 1.3.0
description: Gravatar for Ethereum
features:
  - fullTextSearch
  - nonFatalErrors
dataSources: ...
```

> Note that using a feature without declaring it will incur a **validation error** during Subgraph deployment, but no errors will occur if a feature is declared but not used.

## Timeseries and Aggregations

Prerequisites:

- Subgraph specVersion must be ≥1.1.0.

Timeseries and aggregations enable your Subgraph to track statistics like daily average price, hourly total transfers, and more.

This feature introduces two new types of Subgraph entity. Timeseries entities record data points with timestamps. Aggregation entities perform pre-declared calculations on the timeseries data points on an hourly or daily basis, then store the results for easy access via GraphQL.

### Example Schema

```graphql
type Data @entity(timeseries: true) {
  id: Int8!
  timestamp: Timestamp!
  price: BigDecimal!
}

type Stats @aggregation(intervals: ["hour", "day"], source: "Data") {
  id: Int8!
  timestamp: Timestamp!
  sum: BigDecimal! @aggregate(fn: "sum", arg: "price")
}
```

### How to Define Timeseries and Aggregations

Timeseries entities are defined with `@entity(timeseries: true)` in the GraphQL schema. Every timeseries entity must:

- have a unique ID of the int8 type
- have a timestamp of the Timestamp type
- include data that will be used for calculation by aggregation entities.

These Timeseries entities can be saved in regular trigger handlers, and act as the “raw data” for the aggregation entities.

Aggregation entities are defined with `@aggregation` in the GraphQL schema. Every aggregation entity defines the source from which it will gather data (which must be a timeseries entity), sets the intervals (e.g., hour, day), and specifies the aggregation function it will use (e.g., sum, count, min, max, first, last).

Aggregation entities are automatically calculated on the basis of the specified source at the end of the required interval.

#### Available Aggregation Intervals

- `hour`: sets the timeseries period every hour, on the hour.
- `day`: sets the timeseries period every day, starting and ending at 00:00.

#### Available Aggregation Functions

- `sum`: Total of all values.
- `count`: Number of values.
- `min`: Minimum value.
- `max`: Maximum value.
- `first`: First value in the period.
- `last`: Last value in the period.

#### Example Aggregations Query

```graphql
{
  stats(interval: "hour", where: { timestamp_gt: 1704085200 }) {
    id
    timestamp
    sum
  }
}
```

[Read more](https://github.com/graphprotocol/graph-node/blob/master/docs/aggregations.md) about Timeseries and Aggregations.

## Non-fatal errors

Indexing errors on already synced Subgraphs will, by default, cause the Subgraph to fail and stop syncing. Subgraphs can alternatively be configured to continue syncing in the presence of errors, by ignoring the changes made by the handler which provoked the error. This gives Subgraph authors time to correct their Subgraphs while queries continue to be served against the latest block, though the results might be inconsistent due to the bug that caused the error. Note that some errors are still always fatal. To be non-fatal, the error must be known to be deterministic.

> **Note:** The Graph Network does not yet support non-fatal errors, and developers should not deploy Subgraphs using that functionality to the network via the Studio.

Enabling non-fatal errors requires setting the following feature flag on the Subgraph manifest:

```yaml
specVersion: 1.3.0
description: Gravatar for Ethereum
features:
    - nonFatalErrors
    ...
```

The query must also opt-in to querying data with potential inconsistencies through the `subgraphError` argument. It is also recommended to query `_meta` to check if the Subgraph has skipped over errors, as in the example:

```graphql
foos(first: 100, subgraphError: allow) {
  id
}

_meta {
  hasIndexingErrors
}
```

If the Subgraph encounters an error, that query will return both the data and a graphql error with the message `"indexing_error"`, as in this example response:

```graphql
"data": {
    "foos": [
        {
          "id": "0xdead"
        }
    ],
    "_meta": {
        "hasIndexingErrors": true
    }
},
"errors": [
    {
        "message": "indexing_error"
    }
]
```

## IPFS/Arweave File Data Sources

File data sources are a new Subgraph functionality for accessing off-chain data during indexing in a robust, extendable way. File data sources support fetching files from IPFS and from Arweave.

> This also lays the groundwork for deterministic indexing of off-chain data, as well as the potential introduction of arbitrary HTTP-sourced data.

### Overview

Rather than fetching files "in line" during handler execution, this introduces templates which can be spawned as new data sources for a given file identifier. These new data sources fetch the files, retrying if they are unsuccessful, running a dedicated handler when the file is found.

This is similar to the [existing data source templates](/developing/creating-a-subgraph/#data-source-templates), which are used to dynamically create new chain-based data sources.

> This replaces the existing `ipfs.cat` API

### Upgrade guide

#### Update `graph-ts` and `graph-cli`

File data sources requires graph-ts >=0.29.0 and graph-cli >=0.33.1

#### Add a new entity type which will be updated when files are found

File data sources cannot access or update chain-based entities, but must update file specific entities.

This may mean splitting out fields from existing entities into separate entities, linked together.

Original combined entity:

```graphql
type Token @entity {
  id: ID!
  tokenID: BigInt!
  tokenURI: String!
  externalURL: String!
  ipfsURI: String!
  image: String!
  name: String!
  description: String!
  type: String!
  updatedAtTimestamp: BigInt
  owner: User!
}
```

New, split entity:

```graphql
type Token @entity {
  id: ID!
  tokenID: BigInt!
  tokenURI: String!
  ipfsURI: TokenMetadata
  updatedAtTimestamp: BigInt
  owner: String!
}

type TokenMetadata @entity {
  id: ID!
  image: String!
  externalURL: String!
  name: String!
  description: String!
}
```

If the relationship is 1:1 between the parent entity and the resulting file data source entity, the simplest pattern is to link the parent entity to a resulting file entity by using the IPFS CID as the lookup. Get in touch on Discord if you are having difficulty modelling your new file-based entities!

> You can use [nested filters](/subgraphs/querying/graphql-api/#example-for-nested-entity-filtering) to filter parent entities on the basis of these nested entities.

#### Add a new templated data source with `kind: file/ipfs` or `kind: file/arweave`

This is the data source which will be spawned when a file of interest is identified.

```yaml
templates:
  - name: TokenMetadata
    kind: file/ipfs
    mapping:
      apiVersion: 0.0.9
      language: wasm/assemblyscript
      file: ./src/mapping.ts
      handler: handleMetadata
      entities:
        - TokenMetadata
      abis:
        - name: Token
          file: ./abis/Token.json
```

> Currently `abis` are required, though it is not possible to call contracts from within file data sources

The file data source must specifically mention all the entity types which it will interact with under `entities`. See [limitations](#limitations) for more details.

#### Create a new handler to process files

This handler should accept one `Bytes` parameter, which will be the contents of the file, when it is found, which can then be processed. This will often be a JSON file, which can be processed with `graph-ts` helpers ([documentation](/subgraphs/developing/creating/graph-ts/api/#json-api)).

The CID of the file as a readable string can be accessed via the `dataSource` as follows:

```typescript
const cid = dataSource.stringParam()
```

Example handler:

```typescript
import { json, Bytes, dataSource } from '@graphprotocol/graph-ts'
import { TokenMetadata } from '../generated/schema'

export function handleMetadata(content: Bytes): void {
  let tokenMetadata = new TokenMetadata(dataSource.stringParam())
  const value = json.fromBytes(content).toObject()
  if (value) {
    const image = value.get('image')
    const name = value.get('name')
    const description = value.get('description')
    const externalURL = value.get('external_url')

    if (name && image && description && externalURL) {
      tokenMetadata.name = name.toString()
      tokenMetadata.image = image.toString()
      tokenMetadata.externalURL = externalURL.toString()
      tokenMetadata.description = description.toString()
    }

    tokenMetadata.save()
  }
}
```

#### Spawn file data sources when required

You can now create file data sources during execution of chain-based handlers:

- Import the template from the auto-generated `templates`
- call `TemplateName.create(cid: string)` from within a mapping, where the cid is a valid content identifier for IPFS or Arweave

For IPFS, Graph Node supports [v0 and v1 content identifiers](https://docs.ipfs.tech/concepts/content-addressing/), and content identifiers with directories (e.g. `bafyreighykzv2we26wfrbzkcdw37sbrby4upq7ae3aqobbq7i4er3tnxci/metadata.json`).

For Arweave, as of version 0.33.0 Graph Node can fetch files stored on Arweave based on their [transaction ID](https://docs.arweave.org/developers/arweave-node-server/http-api#transactions) from an Arweave gateway ([example file](https://bdxujjl5ev5eerd5ouhhs6o4kjrs4g6hqstzlci5pf6vhxezkgaa.arweave.net/CO9EpX0lekJEfXUOeXncUmMuG8eEp5WJHXl9U9yZUYA)). Arweave supports transactions uploaded via Irys (previously Bundlr), and Graph Node can also fetch files based on [Irys manifests](https://docs.irys.xyz/overview/gateways#indexing).

Example:

```typescript
import { TokenMetadata as TokenMetadataTemplate } from '../generated/templates'

const ipfshash = 'QmaXzZhcYnsisuue5WRdQDH6FDvqkLQX1NckLqBYeYYEfm'
//This example code is for a Crypto coven Subgraph. The above ipfs hash is a directory with token metadata for all crypto coven NFTs.

export function handleTransfer(event: TransferEvent): void {
  let token = Token.load(event.params.tokenId.toString())
  if (!token) {
    token = new Token(event.params.tokenId.toString())
    token.tokenID = event.params.tokenId

    token.tokenURI = '/' + event.params.tokenId.toString() + '.json'
    const tokenIpfsHash = ipfshash + token.tokenURI
    //This creates a path to the metadata for a single Crypto coven NFT. It concats the directory with "/" + filename + ".json"

    token.ipfsURI = tokenIpfsHash

    TokenMetadataTemplate.create(tokenIpfsHash)
  }

  token.updatedAtTimestamp = event.block.timestamp
  token.owner = event.params.to.toHexString()
  token.save()
}
```

This will create a new file data source, which will poll Graph Node's configured IPFS or Arweave endpoint, retrying if it is not found. When the file is found, the file data source handler will be executed.

This example is using the CID as the lookup between the parent `Token` entity and the resulting `TokenMetadata` entity.

> Previously, this is the point at which a Subgraph developer would have called `ipfs.cat(CID)` to fetch the file

Congratulations, you are using file data sources!

#### Deploying your Subgraphs

You can now `build` and `deploy` your Subgraph to any Graph Node >=v0.30.0-rc.0.

#### Limitations

File data source handlers and entities are isolated from other Subgraph entities, ensuring that they are deterministic when executed, and ensuring no contamination of chain-based data sources. To be specific:

- Entities created by File Data Sources are immutable, and cannot be updated
- File Data Source handlers cannot access entities from other file data sources
- Entities associated with File Data Sources cannot be accessed by chain-based handlers

> While this constraint should not be problematic for most use-cases, it may introduce complexity for some. Please get in touch via Discord if you are having issues modelling your file-based data in a Subgraph!

Additionally, it is not possible to create data sources from a file data source, be it an onchain data source or another file data source. This restriction may be lifted in the future.

#### Best practices

If you are linking NFT metadata to corresponding tokens, use the metadata's IPFS hash to reference a Metadata entity from the Token entity. Save the Metadata entity using the IPFS hash as an ID.

You can use [DataSource context](/subgraphs/developing/creating/graph-ts/api/#entity-and-datasourcecontext) when creating File Data Sources to pass extra information which will be available to the File Data Source handler.

If you have entities which are refreshed multiple times, create unique file-based entities using the IPFS hash & the entity ID, and reference them using a derived field in the chain-based entity.

> We are working to improve the above recommendation, so queries only return the "most recent" version

#### Known issues

File data sources currently require ABIs, even though ABIs are not used ([issue](https://github.com/graphprotocol/graph-cli/issues/961)). Workaround is to add any ABI.

Handlers for File Data Sources cannot be in files which import `eth_call` contract bindings, failing with "unknown import: `ethereum::ethereum.call` has not been defined" ([issue](https://github.com/graphprotocol/graph-node/issues/4309)). Workaround is to create file data source handlers in a dedicated file.

#### Examples

[Crypto Coven Subgraph migration](https://github.com/azf20/cryptocoven-api/tree/file-data-sources-refactor)

#### References

[GIP File Data Sources](https://forum.thegraph.com/t/gip-file-data-sources/2721)

## Indexed Argument Filters / Topic Filters

> **Requires**: [SpecVersion](#specversion-releases) >= `1.2.0`

Topic filters, also known as indexed argument filters, are a powerful feature in Subgraphs that allow users to precisely filter blockchain events based on the values of their indexed arguments.

- These filters help isolate specific events of interest from the vast stream of events on the blockchain, allowing Subgraphs to operate more efficiently by focusing only on relevant data.

- This is useful for creating personal Subgraphs that track specific addresses and their interactions with various smart contracts on the blockchain.

### How Topic Filters Work

When a smart contract emits an event, any arguments that are marked as indexed can be used as filters in a Subgraph's manifest. This allows the Subgraph to listen selectively for events that match these indexed arguments.

- The event's first indexed argument corresponds to `topic1`, the second to `topic2`, and so on, up to `topic3`, since the Ethereum Virtual Machine (EVM) allows up to three indexed arguments per event.

```solidity
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

contract Token {
    // Event declaration with indexed parameters for addresses
    event Transfer(address indexed from, address indexed to, uint256 value);

    // Function to simulate transferring tokens
    function transfer(address to, uint256 value) public {
        // Emitting the Transfer event with from, to, and value
        emit Transfer(msg.sender, to, value);
    }
}
```

In this example:

- The `Transfer` event is used to log transactions of tokens between addresses.
- The `from` and `to` parameters are indexed, allowing event listeners to filter and monitor transfers involving specific addresses.
- The `transfer` function is a simple representation of a token transfer action, emitting the Transfer event whenever it is called.

#### Configuration in Subgraphs

Topic filters are defined directly within the event handler configuration in the Subgraph manifest. Here is how they are configured:

```yaml
eventHandlers:
  - event: SomeEvent(indexed uint256, indexed address, indexed uint256)
    handler: handleSomeEvent
    topic1: ['0xValue1', '0xValue2']
    topic2: ['0xAddress1', '0xAddress2']
    topic3: ['0xValue3']
```

In this setup:

- `topic1` corresponds to the first indexed argument of the event, `topic2` to the second, and `topic3` to the third.
- Each topic can have one or more values, and an event is only processed if it matches one of the values in each specified topic.

#### Filter Logic

- Within a Single Topic: The logic functions as an OR condition. The event will be processed if it matches any one of the listed values in a given topic.
- Between Different Topics: The logic functions as an AND condition. An event must satisfy all specified conditions across different topics to trigger the associated handler.

#### Example 1: Tracking Direct Transfers from Address A to Address B

```yaml
eventHandlers:
  - event: Transfer(indexed address,indexed address,uint256)
    handler: handleDirectedTransfer
    topic1: ['0xAddressA'] # Sender Address
    topic2: ['0xAddressB'] # Receiver Address
```

In this configuration:

- `topic1` is configured to filter `Transfer` events where `0xAddressA` is the sender.
- `topic2` is configured to filter `Transfer` events where `0xAddressB` is the receiver.
- The Subgraph will only index transactions that occur directly from `0xAddressA` to `0xAddressB`.

#### Example 2: Tracking Transactions in Either Direction Between Two or More Addresses

```yaml
eventHandlers:
  - event: Transfer(indexed address,indexed address,uint256)
    handler: handleTransferToOrFrom
    topic1: ['0xAddressA', '0xAddressB', '0xAddressC'] # Sender Address
    topic2: ['0xAddressB', '0xAddressC'] # Receiver Address
```

In this configuration:

- `topic1` is configured to filter `Transfer` events where `0xAddressA`, `0xAddressB`, `0xAddressC` is the sender.
- `topic2` is configured to filter `Transfer` events where `0xAddressB` and `0xAddressC` is the receiver.
- The Subgraph will index transactions that occur in either direction between multiple addresses allowing for comprehensive monitoring of interactions involving all addresses.

## Declared eth_call

Declarative `eth_calls` are a valuable Subgraph feature that allows `eth_calls` to be executed ahead of time, enabling `graph-node` to execute them in parallel.

This feature does the following:

- Significantly improves the performance of fetching data from the Ethereum blockchain by reducing the total time for multiple calls and optimizing the Subgraph's overall efficiency.
- Allows faster data fetching, resulting in quicker query responses and a better user experience.
- Reduces wait times for applications that need to aggregate data from multiple Ethereum calls, making the data retrieval process more efficient.

### Key Concepts

- Declarative `eth_calls`: Ethereum calls that are defined to be executed in parallel rather than sequentially.
- Parallel Execution: Instead of waiting for one call to finish before starting the next, multiple calls can be initiated simultaneously.
- Time Efficiency: The total time taken for all the calls changes from the sum of the individual call times (sequential) to the time taken by the longest call (parallel).

#### Scenario without Declarative `eth_calls`

Imagine you have a Subgraph that needs to make three Ethereum calls to fetch data about a user's transactions, balance, and token holdings.

Traditionally, these calls might be made sequentially:

1.  Call 1 (Transactions): Takes 3 seconds
2.  Call 2 (Balance): Takes 2 seconds
3.  Call 3 (Token Holdings): Takes 4 seconds

Total time taken = 3 + 2 + 4 = 9 seconds

#### Scenario with Declarative `eth_calls`

With this feature, you can declare these calls to be executed in parallel:

1.  Call 1 (Transactions): Takes 3 seconds
2.  Call 2 (Balance): Takes 2 seconds
3.  Call 3 (Token Holdings): Takes 4 seconds

Since these calls are executed in parallel, the total time taken is equal to the time taken by the longest call.

Total time taken = max (3, 2, 4) = 4 seconds

#### How it Works

1.  Declarative Definition: In the Subgraph manifest, you declare the Ethereum calls in a way that indicates they can be executed in parallel.
2.  Parallel Execution Engine: The Graph Node's execution engine recognizes these declarations and runs the calls simultaneously.
3.  Result Aggregation: Once all calls are complete, the results are aggregated and used by the Subgraph for further processing.

#### Example Configuration in Subgraph Manifest

Declared `eth_calls` can access the `event.address` of the underlying event as well as all the `event.params`.

`subgraph.yaml` using `event.address`:

```yaml
eventHandlers:
event: Swap(indexed address,indexed address,int256,int256,uint160,uint128,int24)
handler: handleSwap
calls:
  global0X128: Pool[event.address].feeGrowthGlobal0X128()
  global1X128: Pool[event.address].feeGrowthGlobal1X128()
```

Details for the example above:

- `global0X128` is the declared `eth_call`.
- The text (`global0X128`) is the label for this `eth_call` which is used when logging errors.
- The text (`Pool[event.address].feeGrowthGlobal0X128()`) is the actual `eth_call` that will be executed, which is in the form of `Contract[address].function(arguments)`
- The `address` and `arguments` can be replaced with variables that will be available when the handler is executed.

`subgraph.yaml` using `event.params`

```yaml
calls:
  - ERC20DecimalsToken0: ERC20[event.params.token0].decimals()
```

### Grafting onto Existing Subgraphs

> **Note:** it is not recommended to use grafting when initially upgrading to The Graph Network. Learn more [here](/subgraphs/cookbook/grafting/#important-note-on-grafting-when-upgrading-to-the-network).

When a Subgraph is first deployed, it starts indexing events at the genesis block of the corresponding chain (or at the `startBlock` defined with each data source) In some circumstances; it is beneficial to reuse the data from an existing Subgraph and start indexing at a much later block. This mode of indexing is called _Grafting_. Grafting is, for example, useful during development to get past simple errors in the mappings quickly or to temporarily get an existing Subgraph working again after it has failed.

A Subgraph is grafted onto a base Subgraph when the Subgraph manifest in `subgraph.yaml` contains a `graft` block at the top-level:

```yaml
description: ...
graft:
  base: Qm... # Subgraph ID of base Subgraph
  block: 7345624 # Block number
```

When a Subgraph whose manifest contains a `graft` block is deployed, Graph Node will copy the data of the `base` Subgraph up to and including the given `block` and then continue indexing the new Subgraph from that block on. The base Subgraph must exist on the target Graph Node instance and must have indexed up to at least the given block. Because of this restriction, grafting should only be used during development or during an emergency to speed up producing an equivalent non-grafted Subgraph.

Because grafting copies rather than indexes base data, it is much quicker to get the Subgraph to the desired block than indexing from scratch, though the initial data copy can still take several hours for very large Subgraphs. While the grafted Subgraph is being initialized, the Graph Node will log information about the entity types that have already been copied.

The grafted Subgraph can use a GraphQL schema that is not identical to the one of the base Subgraph, but merely compatible with it. It has to be a valid Subgraph schema in its own right, but may deviate from the base Subgraph's schema in the following ways:

- It adds or removes entity types
- It removes attributes from entity types
- It adds nullable attributes to entity types
- It turns non-nullable attributes into nullable attributes
- It adds values to enums
- It adds or removes interfaces
- It changes for which entity types an interface is implemented

> **[Feature Management](#experimental-features):** `grafting` must be declared under `features` in the Subgraph manifest.
