-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement tree
explain for CsvSink
#15204
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @irenjj ❤️
---- | ||
physical_plan | ||
01)┌───────────────────────────┐ | ||
02)│ DataSinkExec │ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is interesting that the format is not shown in this plan 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's triggered by DataSinkExec
:
DisplayFormatType::TreeRender => self.sink().fmt_as(t, f)
CsvSink
is only a part of the DataSinkExec
format.
A more reasonable output format might be:
DataSinkExec
---------------
sink=csv
filegroup...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree -- can you make a follow on PR for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I created a new PR #15206, and I found that the actual type of the sink can be determined from the file extension.
@@ -666,8 +666,10 @@ impl DisplayAs for CsvSink { | |||
write!(f, ")") | |||
} | |||
DisplayFormatType::TreeRender => { | |||
// TODO: collect info | |||
write!(f, "") | |||
if !self.config.file_groups.is_empty() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I debugged a bit more and it seems the issue is that the actual filename target is in self.config.table_path
rather than self.config_file_groups
However, this way is consisent with the other data sinks
@irenjj is there any chance you can make a ticket / PR to fix this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will fix this issue together in #15112.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there is a bug with how the sinks are being rendered but this is consistent with the other sinks, so I think we can merge it for now
This PR also has new test coverage so I am merging it in and we can improve the display in a future follow on |
Which issue does this PR close?
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?