ITADN
reservoir-data/tap-fedidb
reservoir-data/tap-fedidb · 文件 下载 ZIP
文件最后提交记录最后更新时间
README.md

tap-fedidb

Singer tap for FediDB.org.

Built with the Meltano Tap SDK for Singer Taps.

Capabilities

  • catalog
  • state
  • discover
  • about
  • stream-maps
  • schema-flattening
  • batch

Supported Python Versions

  • 3.12
  • 3.13
  • 3.14

Settings

This tap doesn't have any settings of its own, but it does support the following built-in SDK settings:

SettingRequiredDefaultDescription
stream_mapsFalseNoneConfig object for stream maps capability. For more information check out Stream Maps.
stream_map_configFalseNoneUser-defined config values to be used within map expressions.
faker_configFalseNoneConfig for the Faker instance variable fake used within map expressions. Only applicable if the plugin specifies faker as an additional dependency (through the singer-sdk faker extra or directly).
faker_config.seedFalseNoneValue to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.localeFalseNoneOne or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabledFalseNone'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depthFalseNoneThe max depth to flatten schemas.
batch_configFalseNone
batch_config.encodingFalseNoneSpecifies the format and compression of the batch files.
batch_config.encoding.formatFalseNoneFormat to use for batch files.
batch_config.encoding.compressionFalseNoneCompression format to use for batch files.
batch_config.storageFalseNoneDefines the storage layer to use when writing batch files
batch_config.storage.rootFalseNoneRoot path to use when writing batch files.
batch_config.storage.prefixFalseNonePrefix to use when writing batch files.

A full list of supported settings and capabilities is available by running: tap-fedidb --about

Source Authentication and Authorization

The FediDB API doesn't require authentication at this time.

Usage

You can easily run tap-fedidb by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-fedidb --version
tap-fedidb --help
tap-fedidb --config CONFIG --discover > ./catalog.json

Developer Resources

Initialize your Development Environment

You will need the following tools to develop and test this tap:

  • Python 3.12+
  • uv

Create and Run Tests

Run integration tests:

uvx --with=tox-uv tox -e 3.12

You can also test the tap-fedidb CLI interface directly:

uvx --with=tox-uv tox -e sync

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Your project comes with a custom meltano.yml project file already created. Go ahead and install Meltano if you haven't already.

  1. Install all plugins

    meltano install
  2. Check that the extractor is working properly

    meltano invoke tap-fedidb --version
  3. Execute an ELT pipeline

    meltano run tap-fedidb target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.