Skip to content

fix postcard compatibility for top_hits, add postcard test #2346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 9, 2024
Merged

Conversation

PSeitz
Copy link
Collaborator

@PSeitz PSeitz commented Apr 8, 2024

fixes quickwit-oss/quickwit#4840

Fix intermediate result postcard serialization by removing flatten
Fix intermediate result postcard deserialization by replacing OwnedValue (no self-describing formats)

Delay doc value fetchting to end of segment collection, closes #2347
Fix naming and struct organization

@PSeitz PSeitz requested a review from fulmicoton April 8, 2024 05:04
@@ -92,8 +96,9 @@ pub struct TopHitsVecEntry {

/// Search results, for queries that include field retrieval requests
/// (`docvalue_fields`).
#[serde(flatten)]
pub search_results: FieldRetrivalResult,
Copy link
Collaborator

@fulmicoton fulmicoton Apr 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should have been FieldRetrievalResult too.

Is it still used? Can we remove or rename it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can't be FieldRetrievalResult, since postcard does not support flatten

@@ -344,7 +366,7 @@ impl PartialEq for ComparableDocFeature {
impl Eq for ComparableDocFeature {}

#[derive(Clone, Serialize, Deserialize, Debug)]
struct ComparableDocFeatures(Vec<ComparableDocFeature>, FieldRetrivalResult);
struct ComparableDocFeatures(Vec<ComparableDocFeature>, IntermediateFieldRetrivalResult);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we fix the terrible struct name and remove the 2 positional arguments and use a regular struct?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I did also do a pass on all the names and some comments

@PSeitz PSeitz force-pushed the fix_top_hits branch 3 times, most recently from 38df0c0 to 00752da Compare April 9, 2024 02:49
Copy link
Collaborator

@fulmicoton fulmicoton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ComparableDocFeatures

@PSeitz PSeitz merged commit 92c3297 into main Apr 9, 2024
@PSeitz PSeitz deleted the fix_top_hits branch April 9, 2024 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Top Hits Agg Performance: Delay fetching get_document_field_data in top_hits sending a query with top_hits causes a panic
2 participants