# SharePoint as a second source

How the SharePoint half of this blueprint works, what has to be true before it can be turned on,
and what has to be done by hand. Written after building it once and watching it ingest, so
everything marked **verified** was observed rather than reasoned about.

`EnableSharePointSource` is `true`. The deployed `main` stack indexes the ECE 4960 handouts in
`sites/kb` alongside the CS1112 syllabus in S3 — 25 of 25 documents on the rehearsal, zero failures.
Everything below is what that commits the shared pipeline to, and what has to stay true.

## The decision that governs everything else

Bedrock has two kinds of knowledge base, and only one of them can talk to SharePoint at all.

| | Customer-managed | Managed (**this blueprint**) |
|---|---|---|
| Configuration | `Type: VECTOR` + `VectorKnowledgeBaseConfiguration` + `StorageConfiguration` | `Type: MANAGED`, no storage configuration |
| Vector store | Yours — OpenSearch Serverless, Aurora, Pinecone | Service-managed |
| Data sources | **S3 and Custom only** | S3, SharePoint, Confluence, Web Crawler, Google Drive, OneDrive, Custom |
| Embedding model | You name one | Service-managed |
| Retrieval config | `vectorSearchConfiguration` | `managedSearchConfiguration` |

**SharePoint requires the managed kind.** Attach a SharePoint data source to a customer-managed
knowledge base and creation *succeeds* — the sync is what fails, with
`"secret has an invalid format or missing values"`. That error names the secret and the secret is
fine; Bedrock is failing partway down a path it cannot service. It is worth knowing because the
obvious response is to spend an afternoon rewriting the secret.

This blueprint was already `Type: MANAGED` for cost reasons — see `decisions.md` — so SharePoint
came for free architecturally. The corollary is the reason the choice is not revisitable: a
customer-managed knowledge base stands up an OpenSearch Serverless collection whose OCU floor
bills in the hundreds of USD per month whether or not anything is indexed.

## What owns what

| Layer | Owner | Why |
|---|---|---|
| Knowledge base, both data sources, both verifiers, IAM | `infra/knowledgebase.yml` | CloudFormation reaches the managed connector via `MANAGED_KNOWLEDGE_BASE_CONNECTOR`. |
| Entra app registration, its certificate, its consent | Terraform, `infra/azure/` | CloudFormation cannot reach an Entra tenant. |
| The `.p12`, the secret's *value*, the per-site grant | By hand, once | A certificate cannot be generated by CloudFormation, a secret value never belongs in git, and the per-site grant is a Graph call against a site that may not exist yet. |

Two things about that split are worth stating because they invert what the AWS ecosystem
suggests:

- **Terraform cannot express a managed knowledge base.** The AWS provider offers only
  `vector_knowledge_base_configuration`, i.e. customer-managed, through at least 5.100. So the
  usual "Terraform owns AWS" instinct produces the wrong kind of knowledge base. Here
  CloudFormation owns the AWS side and Terraform is confined to Entra, which is also this repo's
  standing rule.
- **A managed SharePoint connector is not console-only.** `ConnectorParameters` is free-form
  `Json`, so the entire SharePoint connector body is expressible in a CloudFormation template.
  That is what makes this a blueprint rather than a runbook.

## Turning it on

In this order. Each step is unverifiable until the one before it is done.

1. **Entra app registration with a certificate**, consented on **both** resource applications
   (below). No client secret at all — certificate auth means the app should have none.
2. **Per-site grant** for each site in `siteUrls`, if the app holds `Sites.Selected`.
3. **Verify the Entra half standalone** — sign a client assertion, request both audiences, read
   the site. Do this before Bedrock is involved at all.
4. **`.p12` into `SharePointCertificateBucket`** under the key `SharePointCertificateKey` names.
5. **Secret populated** with exactly `clientId` and `certificatePassword`.
6. **Rehearse**: deploy the template by hand with `Environment=test` and
   `EnableSharePointSource=true`. Read the data source status from
   `aws bedrock-agent list-data-sources`, not from the stack — see `warnings.md` for why the stack
   lies.
7. **One PR that flips the flag** in both `infra/knowledgebase.yml` and `pipeline/pipeline.yml`,
   with a `SharePointSmokeQuery` the SharePoint corpus can actually answer.

Step 3 is the one that saves the day. It separates "Entra is misconfigured" from "Bedrock is
misconfigured", and every hour lost building this the first time was lost to conflating them.

## The Entra side

### Two resource applications, not one

The connector talks to Microsoft Graph **and** to the SharePoint REST API. They are separate
resource applications and each needs its own application-permission grant:

| Resource | App id |
|---|---|
| Microsoft Graph | `00000003-0000-0000-c000-000000000000` |
| SharePoint REST | `00000003-0000-0ff1-ce00-000000000000` |

Granting only Graph produces a token that reads site metadata and fails on content. **Verified**:
both audiences returned `roles: ['Sites.Selected']` before the first successful ingestion.

### `Sites.Selected`, not `Sites.Read.All`

`Sites.Read.All` reads every site in the tenant. `Sites.Selected` reads only sites explicitly
granted to the app, and the extra work is one Graph call per site.

**Look up app role ids in your own tenant rather than copying them.** Several are near-identical
by eye, and a wrong GUID silently grants something else — this is exactly how a `Sites.Read.All`
plus `User.Read.All` over-grant happens when `Files.Read.All` was meant.

```sh
az ad sp show --id 00000003-0000-0000-c000-000000000000 \
  --query "appRoles[?value=='Sites.Selected'].{v:value,id:id}" -o json
az ad sp show --id 00000003-0000-0ff1-ce00-000000000000 \
  --query "appRoles[?value=='Sites.Selected'].{v:value,id:id}" -o json
```

Use `-o json`. The table renderer drops the `id` column silently, which reads as the role having
no id rather than as a formatting artifact.

The values these resolved to — first-party app role ids are stable across tenants, but confirm
rather than trust:

| Role | Resource | Id |
|---|---|---|
| `Sites.Selected` | Graph | `883ea226-0bf2-4a8f-9f9d-92c9162a727d` |
| `Sites.Selected` | SharePoint REST | `20d37865-089c-4dee-8c41-6967602d4ac8` |
| `Sites.FullControl.All` | Graph | `a82116e5-55eb-4c41-a434-62fe8a61c773` |

Consent for an application permission is `azuread_app_role_assignment`. It is **not**
`azuread_service_principal_delegated_permission_grant`, which is delegated-only and silently
fails to consent a `type = "Role"` permission. `infra/azure/sharepoint-entra.tf.sample` has the
shape.

### The per-site grant

`Sites.Selected` on its own grants **nothing**. Each site needs an explicit grant, issued by a
principal holding Graph `Sites.FullControl.All`:

```
POST https://graph.microsoft.com/v1.0/sites/{siteId}/permissions
{
  "roles": ["read"],
  "grantedToIdentities": [
    {"application": {"id": "<connector client id>", "displayName": "<app name>"}}
  ]
}
```

Resolve `{siteId}` first with `GET /v1.0/sites/{host}:{server-relative-path}` — e.g.
`/v1.0/sites/contoso.sharepoint.com:/sites/kb` — which returns a compound `host,siteGuid,webGuid`
triple. `GET` the same `/permissions` path to list every grant on a site, which is the check to
run before assuming a permission problem lies elsewhere.

Create the `Sites.FullControl.All` granter app as a throwaway, use it, delete it. **Verified**:
the grant survives the granter's deletion. `sharepoint-entra.tf.sample` gates it behind
`create_granter` so removing it is one flag flip.

### The certificate

```sh
umask 077
mkdir -p .certs && chmod 700 .certs
openssl rand -base64 32 > .certs/p12_password      # write to a file; never echo it
openssl req -x509 -newkey rsa:2048 -nodes -days 730 \
  -keyout .certs/private_key.pem -out .certs/certificate.cer \
  -subj "/CN=bedrock-sharepoint-connector"
openssl pkcs12 -export -out .certs/certificate.p12 \
  -inkey .certs/private_key.pem -in .certs/certificate.cer \
  -passout file:.certs/p12_password
openssl pkcs12 -in .certs/certificate.p12 -nokeys -passin file:.certs/p12_password >/dev/null \
  && echo "p12 opens with stored password"
```

The **public** `.cer` goes on the app registration; the **`.p12`** goes to S3 and its password
goes in the secret. `end_date` on the Entra certificate must be at or before the certificate's own
`notAfter` or Entra rejects it.

`.certs/`, `*.tfstate*` and `terraform.tfvars` must all be gitignored — the `.p12` password lands
in `.certs/` *and* in Terraform state. This repo is public and **has no secret scanning**, so
nothing mechanical would catch a slip.

## The AWS side

**The secret** holds exactly two camelCase keys for `ENTRA_ID_APP_ONLY`:

```json
{"clientId": "...", "certificatePassword": "..."}
```

Wrong key names produce the same generic `"invalid format or missing values"` error as a wrong
knowledge base type, so there is no diagnostic distance between the two mistakes.
(`OAUTH2_APP` instead wants `clientId`, `clientSecret`, `username`, `password` — and an
MFA-exempt human account, which is why this blueprint does not use it.)

**The certificate bucket** is private, SSE-AES256, public access blocked. The connector fetches
the `.p12` itself; it is never passed inline.

**The role.** `KnowledgeBaseRole` gains three statements when the flag is on:
`secretsmanager:GetSecretValue` on the secret, `s3:ListBucket` on the bucket, and `s3:GetObject`
on the certificate **prefix** — not the object. The connector also probes for a sibling
`<key>.metadata.json`, and a grant on the single object turns that probe into a denied call.

No `bedrock:InvokeModel` is needed for any of this: embedding is service-managed. See
`decisions.md` for why the embedding statements are still in the template anyway.

## Verifying the Entra half before Bedrock

Standalone script, run before the flag is flipped and again after the granter app is deleted.
Sign a client assertion with the private key and request both audiences:

- `https://graph.microsoft.com/.default`
- `https://{tenant}.sharepoint.com/.default`

Decode each token and assert the `roles` claim contains what was granted — the roles assigned and
the roles Entra attaches are not always the same set. A correct result is `roles: ['Sites.Selected']`
on both, with `aud` of `https://graph.microsoft.com` and `00000003-0000-0ff1-ce00-000000000000`
respectively. Then read the site and list `/sites/{siteId}/drive/root/children`.

Assertion claims: `aud` = the token endpoint, `iss` = `sub` = client id, plus `jti`, `nbf`, `exp`.
The header carries `x5t` = base64url of the certificate's SHA-1 DER digest.

Two things that cost time here:

- `GET /sites/{host}:{path}:/drive/root/children` returns `"Url specified is invalid."` Resolve
  the site id first, then `GET /sites/{siteId}/drive/root/children`.
- Python < 3.12 rejects backslashes inside f-string expressions, so `f"{d[\"id\"]}"` in a
  `python3 -c` one-liner is a `SyntaxError`. Concatenate instead.

**Mask credentials by value, not by key name.** A filter matching keys containing
`secret`/`password`/`pass` prints `privateKey` in full — and for this auth type that key *is* the
credential. Anything that reaches a terminal or a transcript is compromised; rotate it.

## What the deploy proves, and what it does not

`SharePointIngestionVerifier` is a second instance of the same verifier Lambda, sequenced after
the S3 one with `DependsOn`. It asserts the same five things against the SharePoint data source.

**Proves**: the data source reached `AVAILABLE`, an ingestion job ran to `COMPLETE`, at least one
document was scanned, none failed. Those statistics are per-data-source, so an empty SharePoint
source cannot pass green — which was the open hole the previous version of this blueprint
documented and did not close.

**Does not prove**: that the retrieved chunk came from SharePoint. `bedrock:Retrieve` queries the
whole knowledge base. A `SharePointSmokeQuery` the S3 syllabus can also answer makes the retrieval
assertion pass on S3 content. The current default,
`What does the syllabus say about attendance?`, is **exactly that mistake** and is a placeholder —
replace it with a question only the SharePoint corpus can answer in the PR that flips the flag.

`numberOfNewDocumentsIndexed: 0` on a `COMPLETE` job is a failure wearing a success label. The
verifier fails on zero *scanned* and on any *failed*, and reports indexed to SSM and to the stack
outputs; read `SharePointDocumentsIndexed`, not just the stage colour.

## Changing configuration later

Through this template, a `ConnectorParameters` edit is an ordinary stack update, and the verifier
re-ingests and re-asserts. By hand, `update-data-source` requires the **full** configuration
rather than a patch — read the current config, mutate one field, send it all back, or the
service-filled defaults get silently reset. `--name` is required on update even when unchanged.

**Narrowing scope does not retroactively purge.** Turning `crawlPages` from true to false means
the connector no longer *sees* those pages, so it cannot diff them as deleted. **Verified**: the
post-change sync reported `numberOfDocumentsScanned: 1, numberOfDocumentsDeleted: 0` and the two
`SitePages/*.aspx` documents were still returned by `retrieve` afterwards. Document-level deletion
is not available to fix it either:

```
ValidationException: Invalid data source type [SHAREPOINTV3] provided.
Only S3 and Custom data source supported for document level request.
```

That rejects `list-knowledge-base-documents` too, so the indexed set cannot even be enumerated
that way. The only purge is to delete and recreate the data source, which is why this template
sets `DataDeletionPolicy: DELETE` on the SharePoint source and `RETAIN` on the S3 one.
**Verified** end to end: after recreation with `crawlPages: false` and a fresh sync, the query
that previously matched both pages returned only document-library chunks. Budget ~4 minutes in
`DELETING`, and poll until `get-data-source` fails with a not-found error rather than expecting a
terminal status. **The data source id changes** — the SSM mirror updates itself, anything written
down elsewhere does not.

## Operations

- **Keeping the index fresh is `EnableScheduledSync`, and it does not watch anything.** A SharePoint
  site changes without a PR, so between merges the SharePoint half is the stalest thing in the
  blueprint. The schedule fixes that and reports nothing: Scheduler cannot call `GetIngestionJob` or
  `Retrieve`. In particular it is **not** a certificate-expiry canary — it will hit the expired
  certificate and discard the finding. See `warnings.md`.
- **Certificate expiry is a hard deadline that turns the shared pipeline red.** Rotating means
  regenerating `.certs/`, re-applying Terraform (which replaces both the Entra certificate and the
  S3 object), and re-syncing. Nothing in this repo watches the date. Put it somewhere visible.
- **Adding a site** is not a parameter change. `SharePointSiteUrl` is a single `String` and the
  template renders `siteUrls` as a one-element list, so a second site means editing
  `ConnectorParameters` itself — either a second literal entry or switching the parameter to a
  `CommaDelimitedList` and trusting `!Ref` to marshal into a free-form `Json` block, which is
  **untested**. Either way it is a `ConnectorParameters` edit, so it needs the `Environment=test`
  rehearsal and it can hang the deploy rather than fail it. The Entra side of the same change:
  flip `create_granter` to `true`, apply, grant the new site, flip it back.
- **A stale secret from a failed attempt** deletes with
  `delete-secret --force-delete-without-recovery`, or `recovery_window_in_days = 0` for throwaway
  dev secrets.
- **AWS SSO sessions expire mid-build** and the failure reads as a credential problem
  (`Token has expired and refresh failed`). Re-run `aws sso login` before concluding anything is
  misconfigured.
- **`terraform destroy` on the Entra module leaves the AWS side running**, and deleting the
  CloudFormation stack leaves the Entra app registration and the certificate. Neither half tears
  down the other.

## Doing any of this by hand needs a recent AWS CLI

`MANAGED_KNOWLEDGE_BASE_CONNECTOR` and `--knowledge-base-configuration '{"type":"MANAGED"}'` do
not exist in older v2 builds — 2.17 lacks them, 2.36 has them. The failure looks like a malformed
argument rather than an old CLI.

```sh
aws bedrock-agent create-data-source help | grep -c MANAGED_KNOWLEDGE_BASE_CONNECTOR
```

If that prints `0`, install a current v2 without sudo and without touching the system install:

```sh
curl -sS https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o /tmp/awscliv2.zip
cd /tmp && unzip -qo awscliv2.zip
./aws/install --install-dir "$HOME/.local/aws-cli" --bin-dir "$HOME/.local/bin"
```

A fresh shell may still resolve `aws` to the old one, which is its own confusing half-hour.

This does not apply to the pipeline: `cloudformation-deploy-role` deploys the template, and
CloudFormation's Bedrock support is not the CLI's.

## The evidence behind the defaults

The parameter defaults in `infra/knowledgebase.yml` are the values of a configuration observed
working in this account on 2026-08-04: knowledge base `sharepoint-kb` / `KANPIZQSGD`, data source
`XCKE7WVJVW`, one ingestion job `COMPLETE`, one document scanned, one indexed, none failed, and a
retrieval returning document-library chunks.

Two things that run implies which no document states:

- Managed embedding uses **no** `bedrock:*` permission on the service role. That knowledge base
  ingested and answered with a role holding none.
- `Retrieve` on a managed knowledge base rejects `vectorSearchConfiguration` and tells you to use
  `managedSearchConfiguration`.

And one that ranking implies: **rank 1 is often not the answer.** "How do I submit assignments?"
ranked an academic-integrity chunk above the chunk naming the actual submission system, because
the former repeats the query's vocabulary. Fine when an LLM reads the whole result set,
misleading for anything downstream that trusts the top hit alone. Grouping hits by source URL is
the fastest way to confirm a scope change took effect; a single overall count is not.
