Skip to content

Implement experimental release of replication ops #1673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 16 commits into from

Conversation

tsmith023
Copy link
Collaborator

No description provided.

@tsmith023 tsmith023 requested a review from a team as a code owner May 15, 2025 16:33
Copy link

@orca-security-eu orca-security-eu bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

self,
*,
collection: str,
shard: Optional[str],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this have the default None?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, because of the specific pattern I'm adding in weaviate.replication, it doesn't use the stubs-auto-gen logic instead spelling out the async_.py and sync.py files explicitly, the defaults go in those files with the executor not permitting them. I will remove the default from transfer_type above to make it clearer



class _ReplicationReplicateDetailsReplicaStatus(TypedDict):
state: Literal["REGISTERED", "HYDRATING", "FINALIZING", "DEHYDRATING", "READY", "CANCELLED"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't you just use the enum as type here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are just the types of the API response so are the types of response.json(). The Literal here is for internal use only to avoid serialization. Turning them into enums would just move any error on incompatible types from the public class to this private one

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is more about avoiding duplication - I am sure there will be more states added and then you have to add it in two places (which will be forgotten)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, these TypedDict classes would be auto-generated from the schema.json file, but I don't think such a thing exists. The duplication is a good point, I will simply remove these TypedDict classes and have the serialisation work on raw dict. Either way, it needs runtime tests to ensure correctness anyway despite pyright

@weaviate-git-bot
Copy link

Great to see you again! Thanks for the contribution.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Slack channel?

@databyjp
Copy link
Contributor

databyjp commented May 20, 2025

Hey QQ: does it make sense for this to be under client.cluster.replication? Currently that cluster namespace only has nodes afaik and I wonder if this might make sense there. (Nbd if you think otherwise)

@tsmith023
Copy link
Collaborator Author

tsmith023 commented Jun 26, 2025

does it make sense for this to be under client.cluster.replication

@databyjp I think this is a good idea since that is what it is most relevant to

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 60.18237% with 131 lines in your changes missing coverage. Please review.

Please upload report for BASE (dev/1.31@946ace2). Learn more about missing BASE report.

Files with missing lines Patch % Lines
integration/test_replicate.py 23.91% 70 Missing ⚠️
weaviate/replication/operations/executor.py 36.58% 26 Missing ⚠️
weaviate/replication/base.py 47.82% 12 Missing ⚠️
weaviate/rbac/models.py 79.16% 10 Missing ⚠️
weaviate/replication/models.py 86.88% 8 Missing ⚠️
weaviate/replication/async_.py 86.66% 2 Missing ⚠️
weaviate/replication/sync.py 86.66% 2 Missing ⚠️
weaviate/collections/cluster/executor.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             dev/1.31    #1673   +/-   ##
===========================================
  Coverage            ?   87.43%           
===========================================
  Files               ?      266           
  Lines               ?    18139           
  Branches            ?        0           
===========================================
  Hits                ?    15859           
  Misses              ?     2280           
  Partials            ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tsmith023 tsmith023 changed the title Implement client.replication namespace for all replicate ops Implement experimental release of replication ops Jun 26, 2025
@tsmith023
Copy link
Collaborator Author

Closing in favour of #1708

@tsmith023 tsmith023 closed this Jun 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants