Skip to content

[Improve][HDFS][#1336] Fix the path problem of writing to HDFS. #1337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
FlechazoW merged 1 commit into from
Oct 24, 2022
Merged

[Improve][HDFS][#1336] Fix the path problem of writing to HDFS. #1337

FlechazoW merged 1 commit into from
Oct 24, 2022

Conversation

@zhengchaoken
Copy link
Contributor

@zhengchaoken zhengchaoken commented Oct 23, 2022 •
edited
Loading

Purpose of this pull request

修复写入HDFS后的路径问题。使本地调试时能正常写hdfs。

Which issue you fix

Fixes # (1336).

Checklist:

  • I have executed the 'mvn spotless:apply' command to format my code.
  • I have a meaningful commit message (including the issue id, the template of commit message is '[label-type-#issue-id][fixed-module] a meaningful commit message.')
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have checked my code and corrected any misspellings.
  • My commit is only one. (If there are multiple commits, you can use 'git squash' to compress multiple commits into one.)

@zhengchaoken
Copy link
Contributor Author

zhengchaoken commented Oct 23, 2022

本地调试，没有单元测试用例

FlechazoW
FlechazoW reviewed Oct 24, 2022
View reviewed changes
...nnector-hdfs/src/main/java/com/dtstack/chunjun/connector/hdfs/sink/BaseHdfsOutputFormat.java
}
}

/** 文件分隔符(File.separatorChar)在windows为\，而在linux中为/，在hadoop中路径需要固定为/， */
Copy link
Member

@FlechazoW FlechazoW Oct 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you comment in English?

FlechazoW
FlechazoW reviewed Oct 24, 2022
View reviewed changes
...nnector-hdfs/src/main/java/com/dtstack/chunjun/connector/hdfs/sink/BaseHdfsOutputFormat.java

/** 文件分隔符(File.separatorChar)在windows为\，而在linux中为/，在hadoop中路径需要固定为/， */
protected char getHdfsPathChar() {
// hadoop 文件系统固定为/，避免路径不对，文件写入错误及移动失败
Copy link
Member

@FlechazoW FlechazoW Oct 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

@FlechazoW FlechazoW changed the title fix BUG #1336,修复写入HDFS后的路径问题。 [Improve][HDFS][#1336] Fix the path problem of writing to HDFS. Oct 24, 2022
@FlechazoW FlechazoW linked an issue Oct 24, 2022 that may be closed by this pull request
[Bug] [chunjun-connector-hdfs] hdfswriter路径写死，在windows debug时写入目录路径错误 #1336
Closed
3 tasks
@FlechazoW
Copy link
Member

FlechazoW commented Oct 24, 2022

+1

@FlechazoW FlechazoW merged commit a62b374 into DTStack:master Oct 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FlechazoW FlechazoW FlechazoW left review comments

Assignees

@zhengchaoken zhengchaoken

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug] [chunjun-connector-hdfs] hdfswriter路径写死，在windows debug时写入目录路径错误

2 participants

@zhengchaoken @FlechazoW