按照配置文档配置了kerberos认证的相关信息，但是启动的时候报如下错误：
19:09:28.213 [Sink: hdfswriter (1/1)] ERROR org.apache.flink.streaming.api.functions.sink.DtOutputFormatSinkFunction - Cleanup on error failed.
java.io.IOException: Failed on local exception: java.io.IOException: java.lang.IllegalArgumentException: Kerberos principal name does NOT have the expected hostname part: mis; Host Details : local host is: "Lxx/127.0.0.1"; destination host is: "xxx":8020;

不知道是不是因为跨集群访问的原因；
flink集群：集群A（无kerberos认证）
hdfs集群：集群B（kerberos认证）
配置文件如下：
{
"job": {
"content": [{
"reader": {
"parameter": {
"groupId": "testgroupid002",
"topic": "2019allin_calbrk",
"blankIgnore": true,
"codec": "plain",
"consumerSettings": {
"bootstrap.servers": "xxx.22.64.xx:9092",
"zookeeper.connect": "xxx.22.64.xx:2181",
"auto.commit.interval.ms": "1000",
"auto.offset.reset": "latest"
}
},
"name": "kafka10reader"
},
"writer": {
"parameter": {
"path": "/tmp/xx/flinkx",
"fileName": "testfile",
"hadoopConfig": {
"dfs.ha.namenodes.ns1": "nn1,nn2",
"dfs.namenode.rpc-address.ns1.nn2": "hadoop03:8022",
"dfs.client.failover.proxy.provider.ns1": "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider",
"dfs.namenode.rpc-address.ns1.nn1": "hadoop04:8022",
"dfs.nameservices": "xxx",
"hadoop.security.authorization": "true",
"hadoop.security.authentication": "Kerberos",
"dfs.namenode.kerberos.principal": "mis",
"dfs.namenode.keytab.file": "/home/devuser/project/flinkx/jobs/conf/mis.keytab",
"java.security.krb5.conf": "/home/devuser/project/flinkx/jobs/conf/krb5.conf",
"useLocalFile": true

				},
				"column": [{
					"name": "timestamp",
					"index": 0,
					"type": "string"
				}],
				"defaultFS": "hdfs://xxx",
				"writeMode": "APPEND",
				"fieldDelimiter": ",",
				"fileType": "orc",
				"charsetName": "utf-8"
			},
			"name": "hdfswriter",
			"type": 6
		}
	}],
	"setting": {
		"errorLimit": {
			"record": 1
		},
		"speed": {
			"bytes": 1048576,
			"channel": 1
		}
	}
}

}

如果hadoop生态插件需要配置kerberos，该如何配置呢？ #65

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions