go text/scanner

看go的文档,遇到点问题,希望高手解答一下,谢谢~
问题如下:
Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to:

ScanIdents | ScanInts | SkipComments

上面是文档中说的,然后我就写demo,结果很奇怪啊,就是skipcomments的位设置以后无效,代码如下

 const s = `
// This is scanned code.
if a > 10.0 {
    someParsable = text
}`
    sc := new(scanner.Scanner)
    sc.Init(strings.NewReader(s))
    sc.Mode = scanner.ScanInts | scanner.ScanFloats | scanner.ScanIdents | scanner.SkipComments
    var tok rune
    for tok != scanner.EOF {
        tok = sc.Scan()
        fmt.Println("At position", sc.Pos(), ":", sc.TokenText())
    }

输出结果入下

At position <input>:2:2 : /
At position <input>:2:3 : /
At position <input>:2:8 : This
At position <input>:2:11 : is
At position <input>:2:19 : scanned
At position <input>:2:24 : code
At position <input>:2:25 : .
At position <input>:3:3 : if
At position <input>:3:5 : a
At position <input>:3:7 : >
At position <input>:3:12 : 10.0
At position <input>:3:14 : {
At position <input>:4:17 : someParsable
At position <input>:4:19 : =
At position <input>:4:24 : text
At position <input>:5:2 : }
At position <input>:5:2 :

可以看到这个注视没有被识别,被一个一个字符读取来,我试了下除非压根不设置mod,否则就无法跳过comment了?后来我又设置了ScanComments位,才可以

sc.Mode = scanner.ScanInts | scanner.ScanFloats | scanner.ScanIdents | scanner.SkipComments | scanner.ScanComments

感觉很奇怪啊,这样确实输出结果中没有注释部分了,但是文档上不是这样说的啊,求解答

阅读 3.9k
1 个回答

文档上还有这么一句

if set with ScanComments, comments become white space

可能这才是他想表达的意思吧,乱猜的。。

const (
        ScanIdents     = 1 << -Ident
        ScanInts       = 1 << -Int
        ScanFloats     = 1 << -Float // includes Ints
        ScanChars      = 1 << -Char
        ScanStrings    = 1 << -String
        ScanRawStrings = 1 << -RawString
        ScanComments   = 1 << -Comment
        SkipComments   = 1 << -skipComment // if set with ScanComments, comments become white space
        GoTokens       = ScanIdents | ScanFloats | ScanChars | ScanStrings | ScanRawStrings | ScanComments | SkipComments
)
撰写回答
你尚未登录,登录后可以
  • 和开发者交流问题的细节
  • 关注并接收问题和回答的更新提醒
  • 参与内容的编辑和改进,让解决方法与时俱进
推荐问题