Skip to content

fixix(tinybird): Align MV columns and fix downstream pipe queries #2397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

NagyVikt
Copy link

@NagyVikt NagyVikt commented May 6, 2025

=

Deployment was failing due to mismatches between materialized view
pipe outputs and target datasource schemas, as well as downstream
pipes referencing columns that were no longer being materialized.

  • Aligned columns in dub_click_events_pipe final node with the schema
    of dub_click_events_mv.datasource. Added missing fields like
    region, latitude, longitude, engine, ua, identity_hash,
    and referer_url.
  • Aligned columns in dub_lead_events_pipe final node with the schema
    of dub_lead_events_mv.datasource. Initially added missing fields,
    then removed fields (bot, metadata, engine_version, etc.) that
    were not actually present in the target MV schema according to later
    deployment errors.
  • Aligned columns in dub_sale_events_pipe final node with the schema
    of dub_sale_events_mv.datasource. Added missing fields like
    region, latitude, longitude, engine, ua, and referer_url.
  • Removed references to the metadata column from NODE lead_events
    and NODE sale_events within v2_customer_events.pipe, as this
    column was no longer present in dub_lead_events_mv or
    dub_sale_events_mv.
  • Corrected the UNION ALL structure in NODE endpoint of
    v2_customer_events.pipe to ensure all branches selected the
    same number and order of columns after the removal of metadata.

NagyVikt and others added 2 commits May 6, 2025 06:33
Deployment was failing due to mismatches between materialized view
pipe outputs and target datasource schemas, as well as downstream
pipes referencing columns that were no longer being materialized.

- Aligned columns in `dub_click_events_pipe` final node with the schema
  of `dub_click_events_mv.datasource`. Added missing fields like
  `region`, `latitude`, `longitude`, `engine`, `ua`, `identity_hash`,
  and `referer_url`.
- Aligned columns in `dub_lead_events_pipe` final node with the schema
  of `dub_lead_events_mv.datasource`. Initially added missing fields,
  then removed fields (`bot`, `metadata`, `engine_version`, etc.) that
  were not actually present in the target MV schema according to later
  deployment errors.
- Aligned columns in `dub_sale_events_pipe` final node with the schema
  of `dub_sale_events_mv.datasource`. Added missing fields like
  `region`, `latitude`, `longitude`, `engine`, `ua`, and `referer_url`.
- Removed references to the `metadata` column from `NODE lead_events`
  and `NODE sale_events` within `v2_customer_events.pipe`, as this
  column was no longer present in `dub_lead_events_mv` or
  `dub_sale_events_mv`.
- Corrected the `UNION ALL` structure in `NODE endpoint` of
  `v2_customer_events.pipe` to ensure all branches selected the
  same number and order of columns after the removal of `metadata`.
Copy link

vercel bot commented May 6, 2025

@NagyVikt is attempting to deploy a commit to the Dub Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


NagyVikt seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants