Skip to content

ip transformer修改 - #730

Merged
wonderflow merged 3 commits into
qiniu:masterfrom
redHJ:pdr-7242
Aug 29, 2018
Merged

wonderflow merged 3 commits into
qiniu:masterfrom
redHJ:pdr-7242

Conversation

@redHJ

@redHJ redHJ commented Aug 27, 2018

Copy link
Copy Markdown
Collaborator

@wonderflow
@unknwon
ip transfomer 增加选项 local_enable, 表示是否使用本地ip解析,默认为false,使用服务端解析
如果local_enable,pandora sender处相应修改schema

@redHJ

redHJ commented Aug 27, 2018

Copy link
Copy Markdown
Collaborator Author

已实测

Comment thread transforms/ip/ip.go Outdated
DefaultNoUse: true,
Description: "IP数据库路径(data_path)",
Type: transforms.TransformTypeString,
AdvanceDepend: transforms.LocalEnable,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

根据前端的功能做对应修改:https://jira.qiniu.io/browse/PDR-7261

Comment thread transforms/ip/ip.go Outdated
Required: true,
DefaultNoUse: false,
Description: "使用本地解析",
Type: transforms.TransformTypeBoolean,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

说明一下本地IP解析和服务端解析最大的区别:

  1. 本地解析是用客户自己的IP库,更为灵活。
  2. 服务端解析是固定使用七牛IP库,用户无需提供IP库

Comment thread transforms/ip/ip.go Outdated

func (t *Transformer) Init() error {
if t.Key != "" {
t.LocalEnable = true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

默认还是本地解析好了,不要加这种hack了

@redHJ
redHJ force-pushed the pdr-7242 branch 3 times, most recently from 09587ff to 214b7f8 Compare August 28, 2018 08:34
Comment thread transforms/ip/ip.go Outdated
config := make(map[string]interface{})
config[transforms.KeyType] = Name
config[transforms.LocalEnable] = t.LocalEnable
t.keys = GetKeys(t.Key)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里不用GetKeys了,直接原样给出去,在pandora-go-sdk里面去split,然后转成DSL的结构

Comment thread transforms/ip/ip.go Outdated
transforms.KeyFieldName,
{
KeyName: "data_path",
KeyName: transforms.LocalEnable,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个key的名字不应该叫localEnable了,叫 At 或者 TransformAt

Comment thread mgr/runner.go
}

func setSenderConfig(senderConfig conf.MapConf, serverConfigs []map[string]interface{}) conf.MapConf {
if senderConfig[sender.KeySenderType] != sender.TypePandora {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sender.Type可以作为参数传进来,也许以后别的Sender也要这样

Comment thread mgr/runner.go Outdated
}
autoCreate := senderConfig[sender.KeyPandoraAutoCreate]
localEnable, localEnableOk := serverConfig[transforms.LocalEnable].(string)
if localEnableOk && localEnable == ip.Local {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

localEnable == ip.Server

Comment thread mgr/runner.go Outdated
} else if index := strings.Index(autoCreate, schema); index != -1 {
autoCreate = autoCreate[:index] + autoCreate[index+len(schema):]
}
senderConfig[sender.KeyPandoraAutoCreate] = autoCreate

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我觉得这块逻辑要单独做个函数,叫inject Ip Transformer config,以后别人再加个类似的函数就行了,不需要理解你这块for循环里的细节逻辑

Comment thread mgr/runner.go

if !strings.Contains(autoCreate, KeyIP) {
senderConfig[sender.KeyPandoraAutoCreate] += fmt.Sprintf(",%v ip", key)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里拼接的方式可能要修改,看服务端未来会不会支持嵌套类型指定IP字段

@redHJ
redHJ force-pushed the pdr-7242 branch 3 times, most recently from 7a75786 to 27078fa Compare August 28, 2018 11:14
Comment thread mgr/runner.go Outdated
var err error
for _, serverConfig := range serverConfigs {
keyType, ok := serverConfig[transforms.KeyType].(string)
if !ok || keyType != ip.Name {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不用再pandorakeyType是不是等于ip.Name了,下面会switch

Comment thread mgr/runner.go
return senderConfig, fmt.Errorf("key: %v ip transform key in server doesn't support dot(.)", key)
}
autoCreate := senderConfig[sender.KeyPandoraAutoCreate]
transformAt, transformAtOk := serverConfig[transforms.TransformAt].(string)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

先pandora TransformAtOk为false的情况,直接return

Comment thread mgr/runner.go Outdated
return senderConfig, nil
}

if !strings.Contains(autoCreate, KeyIP) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里不应该判断这个,我可以有多个IP字段通过服务端解析

Comment thread mgr/runner.go Outdated
}

if autoCreate == "" {
senderConfig[sender.KeyPandoraAutoCreate] = fmt.Sprintf("%v ip", key)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

你这里要不直接用字符串拼接更好,而且你专门写了个 KeyIP可以用上。

用fmt的时候,能 %s 尽量不要 %v 哈

@redHJ
redHJ force-pushed the pdr-7242 branch 2 times, most recently from 207c9ce to 91da253 Compare August 28, 2018 12:17
Comment thread mgr/runner.go Outdated
return tags
}

func setSenderConfig(senderConfig conf.MapConf, serverConfigs []map[string]interface{}) (conf.MapConf, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果只允许Pandora Sender,函数名可以明确的写 setPandoraServerConfig

Comment thread transforms/ip/ip.go Outdated
Description: "mmdb格式库使用的语种",
Advance: true,
Description: "要进行Transform变化的键(key)",
ToolTip: "对该字段的值进行transform变换, 服务端不支持嵌套(.)",

@wonderflow wonderflow Aug 28, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

后面加上,“对该字段的值进行transform变换, 服务端转换不支持嵌套(.),请先使用rename,本地转换支持”

@redHJ

redHJ commented Aug 28, 2018

Copy link
Copy Markdown
Collaborator Author

comments都修改了

@wonderflow

Copy link
Copy Markdown
Contributor

LGTM

@wonderflow
wonderflow merged commit e9972a3 into qiniu:master Aug 29, 2018
@redHJ
redHJ deleted the pdr-7242 branch January 24, 2019 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

⌂ 换网址