batching queries with useQueries and batshit

[tbowmo]

We've recently included react-query, into our project. We've also taken batshit in, to do batching of data requests. It was a blast from above when we discovered these tools. However the last couple of days has been not so fun.. After the latest release one customer called in complaining that a specific table view was terrible slow now. Like 40-50 seconds before they could interact with it. The table contains some 32.000 rows of data. I has been problematic in the past but not like this.

While debugging code, over the last two days, I found out that the culprit of it all was in fact useQueries and batshit, or how they operate. My code is something like this:

import { create, keyResolver } from '@yornaath/batshit'
import { useQueries } from '@tanstack/react-query'

const batcher = create({
   fetcher: (ids: string[]) => backend.api.fetch(ids),
   resolver: keyResolver('id'),
})

function useData(ids: string[]) {
  return useQueries({
    queries: ids.map((id) => ({
       queryFn: () => batcher.fetch(id) ?? null
    })
    combine: (data) => data.map((item) => item.data).filter((item) => item !== null)
  }
}

Pretty much textbook example, as done on the react-query documentation site, the issue is that we have 32.000 ids that is sent to the backend, the backend responds with data for the 400 of these. The keyResolver() then looks through the 400 records received, for each and every 32.000 ids that the useData hook is called with, in order to return the correct data record to useQueries for each id.

The data from backend is an array of objects, I've tried converting it to an object with Object.fromEntries(data.map((d) => ([d.id, d]))), and use idResolver from batshit library instead (so that we can skip the "find" method 32.000 times). But doesn't cut it.

Bottom line, it's not very efficient, like it takes 15-20 seconds, and bogs the browser like hell (firefox peaks out at 8G of ram at max, chrome at 4-5G). after 40 seconds memory consumption is down at 4-500Mb

Is there anything that I miss here? Any other packages that I should look at? We could just use our old data fetcher wrapped up in useEffect etc. but it does not cache data like react-query does, with an ease of invalidating data etc.. I really would like to use react-query, so any advice etc. is welcome.

[TkDodo]

can you narrow down if it's react-query or batshit where the performance is lost? Because if it's the latter, I think you'd need file an issue there

[tbowmo]

I'll have to investigate Monday, as I found the culprit late Friday afternoon, just before leaving the office. I do have an idea of some performance trials in my combine methods,

As batshit is a relatively new addition, I thought there would be more people here that might had an idea of alternative ways of doing stuff, getting an arbitrary number of records to fit into a deserialized react query world.

Sometimes it's 32000 records, sometimes it's only a subset of the records we fetch, as the rest have been requested elsewhere in the system, and would already be in react querys cache

[tbowmo]

Further tests, without batshit, it does seem that react-query has some issues with large amounts of data

I've created a simple react query, which is getting a dummy data object. With 25000 ids, it takes 13 seconds before the react component gets any data, with 30000 ids it takes 22 seconds (if it doesn't crash the browser)

function data(id: string): Data {
    return {
        id,
        name: `Sigfried ${id}`
    }
}

function createQuery(id: string): UseQueryOptions<Data | null> {
    return {
        queryKey: ['dummy', id],
        queryFn: async () => {
            return data(id)
        },
        enabled: id !== undefined
    }
}


export function useMulti(ids: string[]) {
    return useQueries({
        queries: ids.map((id) => createQuery(id)),
        combine: (results) => {
            if (results.some((item) => !item.isSuccess)) {
                return
            }
            console.log('useQueries', Date.now(), results.length)
            return results.map((item) => item.data)
        }
    })
}

I'm using a simple vite / react / typescript project to test in, and consume the code with a simple component where I auto generates a bunch of ids:

function App() {
  const ref = useRef(0)
  const stableIds = useMemo((): string[] => Array.from(new Array(25000),(_val,index)=> `${index}` ), [])
  const d = useMulti(stableIds)

  useEffect(() => {
    const prev = Date.now() - ref.current 
    ref.current = Date.now()
    console.log('time', Date.now(), prev,  d?.length)
  }, [d])
  return (
    <>
    {d?.map((item) => (<div key={item?.id}>{item?.name}</div>))}
    </>
  )
}

[TkDodo]

put it in a codesandbox please, then I can try to narrow it down what the culprit is.

[tbowmo]

Sure, there is one here https://codesandbox.io/p/sandbox/react-query-large-queries-set-c5244l

[tbowmo]

Some small tests with the above code

15000 ids : ~4 seconds
20000 ids : ~8 seconds
25000 ids: ~13 seconds
30000 ids: ~22 seconds
35000 ids: crash (sigill in chromium)

Memory consumption explodes as well, when waiting. In my mockup it peaks at 2.5GB before crashing in chromium. At the office we saw that firefox peaked out at 8GB before caving in, and 4GB in chrome (we use windows machines at work, while I use linux/ubuntu at home, so that might cause a difference in memory consumption)

[TkDodo]

seems like most of the time is spent in useQueries. I don't think we've optimized it for such large datasets

[tbowmo]

Then currently I would not use react-query for this particular page. It is bound for increase in size as well, 4000 new entries has been added since march. (We do have a task planned for sometime in the future, to reduce the sheer amount of data we want to present to the end users)

[tbowmo]

We are being hit by this more and more now, where we need to take additional measures against too large queries.. Should we convert this to a bug, to raise awareness?

[Miguel-A-Jara]

I took the time to test locally and update the dependencies in @tbowmo's codesanbox from

"@tanstack/react-query": "5.7.0"

to

"@tanstack/react-query": "^5.77.2"

This seems to still be an ongoing issue almost 2 years later, just wondering if this is something that will be worked on?

[TkDodo]

I’m not working on it.