README: make a definitive decision on v2 import path

This change makes it clear that the v2 import path is github.com/russross/blackfriday/v2, and updates various links accordingly. See https://github.com/russross/blackfriday/issues/587#issuecomment-703393820 for details. This change also converges the README for v1 and v2 to be consistent, as they've started to drift apart. (See PR #675 for the equivalent change to the README on v2 branch.) For #587. GitHub-Pull-Request: #674
Merge pull request #586 from kolyshkin/gomod
2020-10-26 23:46:40 -04:00 · 2020-03-25 09:18:22 -06:00 · 2020-03-23 19:11:51 -07:00 · 2020-02-18 16:49:12 -07:00 · 2020-02-18 16:47:11 -07:00 · 2020-02-18 16:43:38 -07:00
17 changed files with 1148 additions and 327 deletions
--- a/.travis.yml
+++ b/.travis.yml
@ -1,18 +1,18 @@
-# Travis CI (http://travis-ci.org/) is a continuous integration service for
-# open source projects. This file configures it to run unit tests for
-# blackfriday.
-
+sudo: false
 language: go
-
 go:
-    - 1.2
-    - 1.3
-    - 1.4
-    - 1.5
-
+  - "1.9.x"
+  - "1.10.x"
+  - "1.11.x"
+  - tip
+matrix:
+  fast_finish: true
+  allow_failures:
+    - go: tip
 install:
-    - go get -d -t -v ./...
-    - go build -v ./...
-
+  - # Do nothing. This is needed to prevent default install action "go get -t -v ./..." from happening here (we want it to happen inside script step).
 script:
-    - go test -v ./...
+  - go get -t -v ./...
+  - diff -u <(echo -n) <(gofmt -d -s .)
+  - go tool vet .
+  - go test -v -race ./...
--- a/LICENSE.txt
+++ b/LICENSE.txt
@ -1,29 +1,28 @@
 Blackfriday is distributed under the Simplified BSD License:

-> Copyright © 2011 Russ Ross
-> All rights reserved.
->
-> Redistribution and use in source and binary forms, with or without
-> modification, are permitted provided that the following conditions
-> are met:
->
-> 1.  Redistributions of source code must retain the above copyright
->     notice, this list of conditions and the following disclaimer.
->
-> 2.  Redistributions in binary form must reproduce the above
->     copyright notice, this list of conditions and the following
->     disclaimer in the documentation and/or other materials provided with
->     the distribution.
->
-> THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-> "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-> LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
-> FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
-> COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
-> INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
-> BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-> LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-> CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
-> LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
-> ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-> POSSIBILITY OF SUCH DAMAGE.
+Copyright © 2011 Russ Ross
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+1.  Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+2.  Redistributions in binary form must reproduce the above
+    copyright notice, this list of conditions and the following
+    disclaimer in the documentation and/or other materials provided with
+    the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
+ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
--- a/README.md
+++ b/README.md
@ -1,4 +1,6 @@
-Blackfriday [![Build Status](https://travis-ci.org/russross/blackfriday.svg?branch=master)](https://travis-ci.org/russross/blackfriday)
+Blackfriday
+[![Build Status][BuildV2SVG]][BuildV2URL]
+[![PkgGoDev][PkgGoDevV2SVG]][PkgGoDevV2URL]
 ===========

 Blackfriday is a [Markdown][1] processor implemented in [Go][2]. It
@ -8,7 +10,7 @@ punctuation substitutions, etc.), and it is safe for all utf-8
 (unicode) input.

 HTML output is currently supported, along with Smartypants
-extensions. An experimental LaTeX output engine is also included.
+extensions.

 It started as a translation from C of [Sundown][3].

@ -16,64 +18,126 @@ It started as a translation from C of [Sundown][3].
 Installation
 ------------

-Blackfriday is compatible with Go 1. If you are using an older
-release of Go, consider using v1.1 of blackfriday, which was based
-on the last stable release of Go prior to Go 1. You can find it as a
-tagged commit on github.
-
-With Go 1 and git installed:
+Blackfriday is compatible with modern Go releases in module mode.
+With Go installed:

    go get github.com/russross/blackfriday

-will download, compile, and install the package into your `$GOPATH`
-directory hierarchy. Alternatively, you can achieve the same if you
-import it into a project:
+will resolve and add the package to the current development module,
+then build and install it. Alternatively, you can achieve the same
+if you import it in a package:

    import "github.com/russross/blackfriday"

 and `go get` without parameters.

+Old versions of Go and legacy GOPATH mode might work,
+but no effort is made to keep them working.
+
+
+Versions
+--------
+
+Currently maintained and recommended version of Blackfriday is `v2`. It's being
+developed on its own branch: https://github.com/russross/blackfriday/tree/v2 and the
+documentation is available at
+https://pkg.go.dev/github.com/russross/blackfriday/v2.
+
+It is `go get`-able in module mode at `github.com/russross/blackfriday/v2`.
+
+Version 2 offers a number of improvements over v1:
+
+* Cleaned up API
+* A separate call to [`Parse`][4], which produces an abstract syntax tree for
+  the document
+* Latest bug fixes
+* Flexibility to easily add your own rendering extensions
+
+Potential drawbacks:
+
+* Our benchmarks show v2 to be slightly slower than v1. Currently in the
+  ballpark of around 15%.
+* API breakage. If you can't afford modifying your code to adhere to the new API
+  and don't care too much about the new features, v2 is probably not for you.
+* Several bug fixes are trailing behind and still need to be forward-ported to
+  v2. See issue [#348](https://github.com/russross/blackfriday/issues/348) for
+  tracking.
+
+If you are still interested in the legacy `v1`, you can import it from
+`github.com/russross/blackfriday`. Documentation for the legacy v1 can be found
+here: https://pkg.go.dev/github.com/russross/blackfriday.
+
+
 Usage
 -----

+### v1
+
 For basic usage, it is as simple as getting your input into a byte
 slice and calling:

-    output := blackfriday.MarkdownBasic(input)
+```go
+output := blackfriday.MarkdownBasic(input)
+```

 This renders it with no extensions enabled. To get a more useful
 feature set, use this instead:

-    output := blackfriday.MarkdownCommon(input)
+```go
+output := blackfriday.MarkdownCommon(input)
+```
+
+### v2
+
+For the most sensible markdown processing, it is as simple as getting your input
+into a byte slice and calling:
+
+```go
+output := blackfriday.Run(input)
+```
+
+Your input will be parsed and the output rendered with a set of most popular
+extensions enabled. If you want the most basic feature set, corresponding with
+the bare Markdown specification, use:
+
+```go
+output := blackfriday.Run(input, blackfriday.WithNoExtensions())
+```

 ### Sanitize untrusted content

 Blackfriday itself does nothing to protect against malicious content. If you are
-dealing with user-supplied markdown, we recommend running blackfriday's output
-through HTML sanitizer such as
-[Bluemonday](https://github.com/microcosm-cc/bluemonday).
+dealing with user-supplied markdown, we recommend running Blackfriday's output
+through HTML sanitizer such as [Bluemonday][5].

-Here's an example of simple usage of blackfriday together with bluemonday:
+Here's an example of simple usage of Blackfriday together with Bluemonday:

-``` go
+```go
 import (
    "github.com/microcosm-cc/bluemonday"
    "github.com/russross/blackfriday"
 )

 // ...
-unsafe := blackfriday.MarkdownCommon(input)
+unsafe := blackfriday.Run(input)
 html := bluemonday.UGCPolicy().SanitizeBytes(unsafe)
 ```

-### Custom options
+### Custom options, v1

 If you want to customize the set of options, first get a renderer
-(currently either the HTML or LaTeX output engines), then use it to
+(currently only the HTML output engine), then use it to
 call the more general `Markdown` function. For examples, see the
 implementations of `MarkdownBasic` and `MarkdownCommon` in
 `markdown.go`.

+### Custom options, v2
+
+If you want to customize the set of options, use `blackfriday.WithExtensions`,
+`blackfriday.WithRenderer` and `blackfriday.WithRefOverride`.
+
+### `blackfriday-tool`
+
 You can also check out `blackfriday-tool` for a more complete example
 of how to use it. Download and install it using:

@ -84,7 +148,7 @@ markdown file using a standalone program.  You can also browse the
 source directly on github if you are just looking for some example
 code:

-* <http://github.com/russross/blackfriday-tool>
+* <https://github.com/russross/blackfriday-tool>

 Note that if you have not already done so, installing
 `blackfriday-tool` will be sufficient to download and install
@ -93,6 +157,22 @@ installed in `$GOPATH/bin`.  This is a statically-linked binary that
 can be copied to wherever you need it without worrying about
 dependencies and library versions.

+### Sanitized anchor names
+
+Blackfriday includes an algorithm for creating sanitized anchor names
+corresponding to a given input text. This algorithm is used to create
+anchors for headings when `EXTENSION_AUTO_HEADER_IDS` is enabled. The
+algorithm has a specification, so that other packages can create
+compatible anchor names and links to those anchors.
+
+The specification is located at https://pkg.go.dev/github.com/russross/blackfriday#hdr-Sanitized_Anchor_Names.
+
+[`SanitizedAnchorName`](https://pkg.go.dev/github.com/russross/blackfriday#SanitizedAnchorName) exposes this functionality, and can be used to
+create compatible links to the anchor names generated by blackfriday.
+This algorithm is also implemented in a small standalone package at
+[`github.com/shurcooL/sanitized_anchor_name`](https://pkg.go.dev/github.com/shurcooL/sanitized_anchor_name). It can be useful for clients
+that want a small package and don't need full functionality of blackfriday.
+

 Features
 --------
@ -114,7 +194,7 @@ All features of Sundown are supported, including:
    know and send me the input that does it.

    NOTE: "safety" in this context means *runtime safety only*. In order to
-    protect yourself agains JavaScript injection in untrusted content, see
+    protect yourself against JavaScript injection in untrusted content, see
    [this example](https://github.com/russross/blackfriday#sanitize-untrusted-content).

 *   **Fast processing**. It is fast enough to render on-demand in
@ -160,7 +240,7 @@ implements the following extensions:
    and supply a language (to make syntax highlighting simple). Just
    mark it like this:

-        ``` go
+        ```go
        func getTrue() bool {
            return true
        }
@ -169,12 +249,21 @@ implements the following extensions:
    You can use 3 or more backticks to mark the beginning of the
    block, and the same number to mark the end of the block.

+    To preserve classes of fenced code blocks while using the bluemonday
+    HTML sanitizer, use the following policy:
+
+    ```go
+    p := bluemonday.UGCPolicy()
+    p.AllowAttrs("class").Matching(regexp.MustCompile("^language-[a-zA-Z0-9]+$")).OnElements("code")
+    html := p.SanitizeBytes(unsafe)
+    ```
+
 *   **Definition lists**. A simple definition list is made of a single-line
    term followed by a colon and the definition for that term.

        Cat
        : Fluffy animal everyone likes
-        
+
        Internet
        : Vector of transmission for pictures of cats

@ -185,7 +274,7 @@ implements the following extensions:
    end of the document. A footnote looks like this:

        This is a footnote.[^1]
-        
+
        [^1]: the footnote text.

 *   **Autolinking**. Blackfriday can find URLs that have not been
@ -222,9 +311,9 @@ Other renderers
 Blackfriday is structured to allow alternative rendering engines. Here
 are a few of note:

-*   [github_flavored_markdown](https://godoc.org/github.com/shurcooL/github_flavored_markdown):
+*   [github_flavored_markdown](https://pkg.go.dev/github.com/shurcooL/github_flavored_markdown):
    provides a GitHub Flavored Markdown renderer with fenced code block
-    highlighting, clickable header anchor links.
+    highlighting, clickable heading anchor links.

    It's not customizable, and its goal is to produce HTML output
    equivalent to the [GitHub Markdown API endpoint](https://developer.github.com/v3/markdown/#render-a-markdown-document-in-raw-mode),
@ -233,25 +322,28 @@ are a few of note:
 *   [markdownfmt](https://github.com/shurcooL/markdownfmt): like gofmt,
    but for markdown.

-*   LaTeX output: renders output as LaTeX. This is currently part of the
-    main Blackfriday repository, but may be split into its own project
-    in the future. If you are interested in owning and maintaining the
-    LaTeX output component, please be in touch.
+*   [LaTeX output](https://gitlab.com/ambrevar/blackfriday-latex):
+    renders output as LaTeX.

-    It renders some basic documents, but is only experimental at this
-    point. In particular, it does not do any inline escaping, so input
-    that happens to look like LaTeX code will be passed through without
-    modification.
+*   [bfchroma](https://github.com/Depado/bfchroma/): provides convenience
+    integration with the [Chroma](https://github.com/alecthomas/chroma) code
+    highlighting library. bfchroma is only compatible with v2 of Blackfriday and
+    provides a drop-in renderer ready to use with Blackfriday, as well as
+    options and means for further customization.
+
+*   [Blackfriday-Confluence](https://github.com/kentaro-m/blackfriday-confluence): provides a [Confluence Wiki Markup](https://confluence.atlassian.com/doc/confluence-wiki-markup-251003035.html) renderer.
+
+*   [Blackfriday-Slack](https://github.com/karriereat/blackfriday-slack): converts markdown to slack message style


-Todo
+TODO
 ----

 *   More unit testing
-*   Improve unicode support. It does not understand all unicode
+*   Improve Unicode support. It does not understand all Unicode
    rules (about what constitutes a letter, a punctuation symbol,
    etc.), so it may fail to detect word boundaries correctly in
-    some instances. It is safe on all utf-8 input.
+    some instances. It is safe on all UTF-8 input.


 License
@ -260,6 +352,13 @@ License
 [Blackfriday is distributed under the Simplified BSD License](LICENSE.txt)


-   [1]: http://daringfireball.net/projects/markdown/ "Markdown"
-   [2]: http://golang.org/ "Go Language"
+   [1]: https://daringfireball.net/projects/markdown/ "Markdown"
+   [2]: https://golang.org/ "Go Language"
   [3]: https://github.com/vmg/sundown "Sundown"
+   [4]: https://pkg.go.dev/github.com/russross/blackfriday/v2#Parse "Parse func"
+   [5]: https://github.com/microcosm-cc/bluemonday "Bluemonday"
+
+   [BuildV2SVG]: https://travis-ci.org/russross/blackfriday.svg?branch=v2
+   [BuildV2URL]: https://travis-ci.org/russross/blackfriday
+   [PkgGoDevV2SVG]: https://pkg.go.dev/badge/github.com/russross/blackfriday/v2
+   [PkgGoDevV2URL]: https://pkg.go.dev/github.com/russross/blackfriday/v2
--- a/block.go
+++ b/block.go
@ -15,8 +15,8 @@ package blackfriday

 import (
 	"bytes"
-
-	"github.com/shurcooL/sanitized_anchor_name"
+	"strings"
+	"unicode"
 )

 // Parse block-level data.
@ -93,7 +93,7 @@ func (p *parser) block(out *bytes.Buffer, data []byte) {

 		// fenced code block:
 		//
-		// ``` go
+		// ``` go info string here
 		// func fact(n int) int {
 		//     if n <= 1 {
 		//         return n
@ -102,7 +102,7 @@ func (p *parser) block(out *bytes.Buffer, data []byte) {
 		// }
 		// ```
 		if p.flags&EXTENSION_FENCED_CODE != 0 {
-			if i := p.fencedCode(out, data, true); i > 0 {
+			if i := p.fencedCodeBlock(out, data, true); i > 0 {
 				data = data[i:]
 				continue
 			}
@ -243,7 +243,7 @@ func (p *parser) prefixHeader(out *bytes.Buffer, data []byte) int {
 	}
 	if end > i {
 		if id == "" && p.flags&EXTENSION_AUTO_HEADER_IDS != 0 {
-			id = sanitized_anchor_name.Create(string(data[i:end]))
+			id = SanitizedAnchorName(string(data[i:end]))
 		}
 		work := func() bool {
 			p.inline(out, data[i:end])
@ -320,6 +320,11 @@ func (p *parser) html(out *bytes.Buffer, data []byte, doRender bool) int {
 			return size
 		}

+		// check for HTML CDATA
+		if size := p.htmlCDATA(out, data, doRender); size > 0 {
+			return size
+		}
+
 		// no special case recognized
 		return 0
 	}
@ -397,12 +402,10 @@ func (p *parser) html(out *bytes.Buffer, data []byte, doRender bool) int {
 	return i
 }

-// HTML comment, lax form
-func (p *parser) htmlComment(out *bytes.Buffer, data []byte, doRender bool) int {
-	i := p.inlineHtmlComment(out, data)
-	// needs to end with a blank line
-	if j := p.isEmpty(data[i:]); j > 0 {
-		size := i + j
+func (p *parser) renderHTMLBlock(out *bytes.Buffer, data []byte, start int, doRender bool) int {
+	// html block needs to end with a blank line
+	if i := p.isEmpty(data[start:]); i > 0 {
+		size := start + i
 		if doRender {
 			// trim trailing newlines
 			end := size
@ -416,6 +419,35 @@ func (p *parser) htmlComment(out *bytes.Buffer, data []byte, doRender bool) int
 	return 0
 }

+// HTML comment, lax form
+func (p *parser) htmlComment(out *bytes.Buffer, data []byte, doRender bool) int {
+	i := p.inlineHTMLComment(out, data)
+	return p.renderHTMLBlock(out, data, i, doRender)
+}
+
+// HTML CDATA section
+func (p *parser) htmlCDATA(out *bytes.Buffer, data []byte, doRender bool) int {
+	const cdataTag = "<![cdata["
+	const cdataTagLen = len(cdataTag)
+	if len(data) < cdataTagLen+1 {
+		return 0
+	}
+	if !bytes.Equal(bytes.ToLower(data[:cdataTagLen]), []byte(cdataTag)) {
+		return 0
+	}
+	i := cdataTagLen
+	// scan for an end-of-comment marker, across lines if necessary
+	for i < len(data) && !(data[i-2] == ']' && data[i-1] == ']' && data[i] == '>') {
+		i++
+	}
+	i++
+	// no end-of-comment marker
+	if i >= len(data) {
+		return 0
+	}
+	return p.renderHTMLBlock(out, data, i, doRender)
+}
+
 // HR, which is the only self-closing block tag considered
 func (p *parser) htmlHr(out *bytes.Buffer, data []byte, doRender bool) int {
 	if data[0] != '<' || (data[1] != 'h' && data[1] != 'H') || (data[2] != 'r' && data[2] != 'R') {
@ -432,19 +464,7 @@ func (p *parser) htmlHr(out *bytes.Buffer, data []byte, doRender bool) int {
 	}

 	if data[i] == '>' {
-		i++
-		if j := p.isEmpty(data[i:]); j > 0 {
-			size := i + j
-			if doRender {
-				// trim newlines
-				end := size
-				for end > 0 && data[end-1] == '\n' {
-					end--
-				}
-				p.r.BlockHtml(out, data[:end])
-			}
-			return size
-		}
+		return p.renderHTMLBlock(out, data, i+1, doRender)
 	}

 	return 0
@ -495,7 +515,7 @@ func (p *parser) htmlFindEnd(tag string, data []byte) int {
 	return i + skip
 }

-func (p *parser) isEmpty(data []byte) int {
+func (*parser) isEmpty(data []byte) int {
 	// it is okay to call isEmpty on an empty buffer
 	if len(data) == 0 {
 		return 0
@ -510,7 +530,7 @@ func (p *parser) isEmpty(data []byte) int {
 	return i + 1
 }

-func (p *parser) isHRule(data []byte) bool {
+func (*parser) isHRule(data []byte) bool {
 	i := 0

 	// skip up to three spaces
@ -539,21 +559,24 @@ func (p *parser) isHRule(data []byte) bool {
 	return n >= 3
 }

-func (p *parser) isFencedCode(data []byte, syntax **string, oldmarker string) (skip int, marker string) {
+// isFenceLine checks if there's a fence line (e.g., ``` or ``` go) at the beginning of data,
+// and returns the end index if so, or 0 otherwise. It also returns the marker found.
+// If syntax is not nil, it gets set to the syntax specified in the fence line.
+// A final newline is mandatory to recognize the fence line, unless newlineOptional is true.
+func isFenceLine(data []byte, info *string, oldmarker string, newlineOptional bool) (end int, marker string) {
 	i, size := 0, 0
-	skip = 0

 	// skip up to three spaces
 	for i < len(data) && i < 3 && data[i] == ' ' {
 		i++
 	}
-	if i >= len(data) {
-		return
-	}

 	// check for the marker characters: ~ or `
+	if i >= len(data) {
+		return 0, ""
+	}
 	if data[i] != '~' && data[i] != '`' {
-		return
+		return 0, ""
 	}

 	c := data[i]
@ -564,79 +587,87 @@ func (p *parser) isFencedCode(data []byte, syntax **string, oldmarker string) (s
 		i++
 	}

-	if i >= len(data) {
-		return
-	}
-
 	// the marker char must occur at least 3 times
 	if size < 3 {
-		return
+		return 0, ""
 	}
 	marker = string(data[i-size : i])

 	// if this is the end marker, it must match the beginning marker
 	if oldmarker != "" && marker != oldmarker {
-		return
+		return 0, ""
 	}

-	if syntax != nil {
-		syn := 0
+	// TODO(shurcooL): It's probably a good idea to simplify the 2 code paths here
+	// into one, always get the info string, and discard it if the caller doesn't care.
+	if info != nil {
+		infoLength := 0
 		i = skipChar(data, i, ' ')

 		if i >= len(data) {
-			return
+			if newlineOptional && i == len(data) {
+				return i, marker
+			}
+			return 0, ""
 		}

-		syntaxStart := i
+		infoStart := i

 		if data[i] == '{' {
 			i++
-			syntaxStart++
+			infoStart++

 			for i < len(data) && data[i] != '}' && data[i] != '\n' {
-				syn++
+				infoLength++
 				i++
 			}

 			if i >= len(data) || data[i] != '}' {
-				return
+				return 0, ""
 			}

 			// strip all whitespace at the beginning and the end
 			// of the {} block
-			for syn > 0 && isspace(data[syntaxStart]) {
-				syntaxStart++
-				syn--
+			for infoLength > 0 && isspace(data[infoStart]) {
+				infoStart++
+				infoLength--
 			}

-			for syn > 0 && isspace(data[syntaxStart+syn-1]) {
-				syn--
+			for infoLength > 0 && isspace(data[infoStart+infoLength-1]) {
+				infoLength--
 			}

 			i++
 		} else {
-			for i < len(data) && !isspace(data[i]) {
-				syn++
+			for i < len(data) && !isverticalspace(data[i]) {
+				infoLength++
 				i++
 			}
 		}

-		language := string(data[syntaxStart : syntaxStart+syn])
-		*syntax = &language
+		*info = strings.TrimSpace(string(data[infoStart : infoStart+infoLength]))
 	}

 	i = skipChar(data, i, ' ')
-	if i >= len(data) || data[i] != '\n' {
-		return
+	if i >= len(data) {
+		if newlineOptional {
+			return i, marker
+		}
+		return 0, ""
+	}
+	if data[i] == '\n' {
+		i++ // Take newline into account
 	}

-	skip = i + 1
-	return
+	return i, marker
 }

-func (p *parser) fencedCode(out *bytes.Buffer, data []byte, doRender bool) int {
-	var lang *string
-	beg, marker := p.isFencedCode(data, &lang, "")
+// fencedCodeBlock returns the end index if data contains a fenced code block at the beginning,
+// or 0 otherwise. It writes to out if doRender is true, otherwise it has no side effects.
+// If doRender is true, a final newline is mandatory to recognize the fenced code block.
+func (p *parser) fencedCodeBlock(out *bytes.Buffer, data []byte, doRender bool) int {
+	var infoString string
+	beg, marker := isFenceLine(data, &infoString, "", false)
 	if beg == 0 || beg >= len(data) {
 		return 0
 	}
@ -647,7 +678,8 @@ func (p *parser) fencedCode(out *bytes.Buffer, data []byte, doRender bool) int {
 		// safe to assume beg < len(data)

 		// check for the end of the code block
-		fenceEnd, _ := p.isFencedCode(data[beg:], nil, marker)
+		newlineOptional := !doRender
+		fenceEnd, _ := isFenceLine(data[beg:], nil, marker, newlineOptional)
 		if fenceEnd != 0 {
 			beg += fenceEnd
 			break
@ -668,13 +700,8 @@ func (p *parser) fencedCode(out *bytes.Buffer, data []byte, doRender bool) int {
 		beg = end
 	}

-	syntax := ""
-	if lang != nil {
-		syntax = *lang
-	}
-
 	if doRender {
-		p.r.BlockCode(out, work.Bytes(), syntax)
+		p.r.BlockCode(out, work.Bytes(), infoString)
 	}

 	return beg
@ -914,7 +941,7 @@ func (p *parser) quote(out *bytes.Buffer, data []byte) int {
 		// irregardless of any contents inside it
 		for data[end] != '\n' {
 			if p.flags&EXTENSION_FENCED_CODE != 0 {
-				if i := p.fencedCode(out, data[end:], false); i > 0 {
+				if i := p.fencedCodeBlock(out, data[end:], false); i > 0 {
 					// -1 to compensate for the extra end++ after the loop:
 					end += i - 1
 					break
@ -1109,6 +1136,15 @@ func (p *parser) listItem(out *bytes.Buffer, data []byte, flags *int) int {
 		i++
 	}

+	// process the following lines
+	containsBlankLine := false
+	sublist := 0
+	codeBlockMarker := ""
+	if p.flags&EXTENSION_FENCED_CODE != 0 && i > line {
+		// determine if codeblock starts on the first line
+		_, codeBlockMarker = isFenceLine(data[line:i], nil, "", false)
+	}
+
 	// get working buffer
 	var raw bytes.Buffer

@ -1116,10 +1152,6 @@ func (p *parser) listItem(out *bytes.Buffer, data []byte, flags *int) int {
 	raw.Write(data[line:i])
 	line = i

-	// process the following lines
-	containsBlankLine := false
-	sublist := 0
-
 gatherlines:
 	for line < len(data) {
 		i++
@ -1128,11 +1160,11 @@ gatherlines:
 		for data[i-1] != '\n' {
 			i++
 		}
-
 		// if it is an empty line, guess that it is part of this item
 		// and move on to the next line
 		if p.isEmpty(data[line:i]) > 0 {
 			containsBlankLine = true
+			raw.Write(data[line:i])
 			line = i
 			continue
 		}
@ -1145,6 +1177,28 @@ gatherlines:

 		chunk := data[line+indent : i]

+		if p.flags&EXTENSION_FENCED_CODE != 0 {
+			// determine if in or out of codeblock
+			// if in codeblock, ignore normal list processing
+			_, marker := isFenceLine(chunk, nil, codeBlockMarker, false)
+			if marker != "" {
+				if codeBlockMarker == "" {
+					// start of codeblock
+					codeBlockMarker = marker
+				} else {
+					// end of codeblock.
+					*flags |= LIST_ITEM_CONTAINS_BLOCK
+					codeBlockMarker = ""
+				}
+			}
+			// we are in a codeblock, write line, and continue
+			if codeBlockMarker != "" || marker != "" {
+				raw.Write(data[line+indent : i])
+				line = i
+				continue gatherlines
+			}
+		}
+
 		// evaluate how this line fits in
 		switch {
 		// is this a nested list item?
@ -1153,6 +1207,14 @@ gatherlines:
 			p.dliPrefix(chunk) > 0:

 			if containsBlankLine {
+				// end the list if the type changed after a blank line
+				if indent <= itemIndent &&
+					((*flags&LIST_TYPE_ORDERED != 0 && p.uliPrefix(chunk) > 0) ||
+						(*flags&LIST_TYPE_ORDERED == 0 && p.oliPrefix(chunk) > 0)) {
+
+					*flags |= LIST_ITEM_END_OF_LIST
+					break gatherlines
+				}
 				*flags |= LIST_ITEM_CONTAINS_BLOCK
 			}

@ -1200,17 +1262,10 @@ gatherlines:

 		// a blank line means this should be parsed as a block
 		case containsBlankLine:
-			raw.WriteByte('\n')
 			*flags |= LIST_ITEM_CONTAINS_BLOCK
 		}

-		// if this line was preceeded by one or more blanks,
-		// re-introduce the blank into the buffer
-		if containsBlankLine {
-			containsBlankLine = false
-			raw.WriteByte('\n')
-
-		}
+		containsBlankLine = false

 		// add the line into the working buffer without prefix
 		raw.Write(data[line+indent : i])
@ -1218,6 +1273,12 @@ gatherlines:
 		line = i
 	}

+	// If reached end of data, the Renderer.ListItem call we're going to make below
+	// is definitely the last in the list.
+	if line >= len(data) {
+		*flags |= LIST_ITEM_END_OF_LIST
+	}
+
 	rawBytes := raw.Bytes()

 	// render the contents of the list item
@ -1332,7 +1393,7 @@ func (p *parser) paragraph(out *bytes.Buffer, data []byte) int {

 				id := ""
 				if p.flags&EXTENSION_AUTO_HEADER_IDS != 0 {
-					id = sanitized_anchor_name.Create(string(data[prev:eol]))
+					id = SanitizedAnchorName(string(data[prev:eol]))
 				}

 				p.r.Header(out, work, level, id)
@ -1362,7 +1423,7 @@ func (p *parser) paragraph(out *bytes.Buffer, data []byte) int {

 		// if there's a fenced code block, paragraph is over
 		if p.flags&EXTENSION_FENCED_CODE != 0 {
-			if p.fencedCode(out, current, false) > 0 {
+			if p.fencedCodeBlock(out, current, false) > 0 {
 				p.renderParagraph(out, data[:i])
 				return i
 			}
@ -1396,3 +1457,24 @@ func (p *parser) paragraph(out *bytes.Buffer, data []byte) int {
 	p.renderParagraph(out, data[:i])
 	return i
 }
+
+// SanitizedAnchorName returns a sanitized anchor name for the given text.
+//
+// It implements the algorithm specified in the package comment.
+func SanitizedAnchorName(text string) string {
+	var anchorName []rune
+	futureDash := false
+	for _, r := range text {
+		switch {
+		case unicode.IsLetter(r) || unicode.IsNumber(r):
+			if futureDash && len(anchorName) > 0 {
+				anchorName = append(anchorName, '-')
+			}
+			futureDash = false
+			anchorName = append(anchorName, unicode.ToLower(r))
+		default:
+			futureDash = true
+		}
+	}
+	return string(anchorName)
+}
--- a/block_test.go
+++ b/block_test.go
@ -657,6 +657,9 @@ func TestUnorderedList(t *testing.T) {
 		"Paragraph\n\n* Linebreak\n",
 		"<p>Paragraph</p>\n\n<ul>\n<li>Linebreak</li>\n</ul>\n",

+		"*   List\n\n1. Spacer Mixed listing\n",
+		"<ul>\n<li>List</li>\n</ul>\n\n<ol>\n<li>Spacer Mixed listing</li>\n</ol>\n",
+
 		"*   List\n    * Nested list\n",
 		"<ul>\n<li>List\n\n<ul>\n<li>Nested list</li>\n</ul></li>\n</ul>\n",

@ -694,18 +697,49 @@ func TestUnorderedList(t *testing.T) {
 		"* List\n        extra indent, same paragraph\n",
 		"<ul>\n<li>List\n    extra indent, same paragraph</li>\n</ul>\n",

-		"* List\n\n        code block\n",
-		"<ul>\n<li><p>List</p>\n\n<pre><code>code block\n</code></pre></li>\n</ul>\n",
+		"* List\n\n        code block\n\n* List continues",
+		"<ul>\n<li><p>List</p>\n\n<pre><code>code block\n</code></pre></li>\n\n<li><p>List continues</p></li>\n</ul>\n",

 		"* List\n\n          code block with spaces\n",
 		"<ul>\n<li><p>List</p>\n\n<pre><code>  code block with spaces\n</code></pre></li>\n</ul>\n",

 		"* List\n\n    * sublist\n\n    normal text\n\n    * another sublist\n",
 		"<ul>\n<li><p>List</p>\n\n<ul>\n<li>sublist</li>\n</ul>\n\n<p>normal text</p>\n\n<ul>\n<li>another sublist</li>\n</ul></li>\n</ul>\n",
+
+		`* Foo
+
+        bar
+
+        qux
+`,
+		`<ul>
+<li><p>Foo</p>
+
+<pre><code>bar
+
+qux
+</code></pre></li>
+</ul>
+`,
 	}
 	doTestsBlock(t, tests, 0)
 }

+func TestFencedCodeBlockWithinList(t *testing.T) {
+	doTestsBlock(t, []string{
+		"* Foo\n\n    ```\n    bar\n\n    qux\n    ```\n",
+		`<ul>
+<li><p>Foo</p>
+
+<pre><code>bar
+
+qux
+</code></pre></li>
+</ul>
+`,
+	}, EXTENSION_FENCED_CODE)
+}
+
 func TestOrderedList(t *testing.T) {
 	var tests = []string{
 		"1. Hello\n",
@ -784,6 +818,12 @@ func TestOrderedList(t *testing.T) {
 		"1. List\n\n          code block with spaces\n",
 		"<ol>\n<li><p>List</p>\n\n<pre><code>  code block with spaces\n</code></pre></li>\n</ol>\n",

+		"1. List\n\n* Spacer Mixed listing\n",
+		"<ol>\n<li>List</li>\n</ol>\n\n<ul>\n<li>Spacer Mixed listing</li>\n</ul>\n",
+
+		"1. List\n* Mixed listing\n",
+		"<ol>\n<li>List</li>\n<li>Mixed listing</li>\n</ol>\n",
+
 		"1. List\n    * Mixted list\n",
 		"<ol>\n<li>List\n\n<ul>\n<li>Mixted list</li>\n</ul></li>\n</ol>\n",

@ -798,6 +838,26 @@ func TestOrderedList(t *testing.T) {

 		"1. numbers\n1. are ignored\n",
 		"<ol>\n<li>numbers</li>\n<li>are ignored</li>\n</ol>\n",
+
+		`1. Foo
+
+        bar
+
+
+
+        qux
+`,
+		`<ol>
+<li><p>Foo</p>
+
+<pre><code>bar
+
+
+
+qux
+</code></pre></li>
+</ol>
+`,
 	}
 	doTestsBlock(t, tests, 0)
 }
@ -900,6 +960,14 @@ func TestDefinitionList(t *testing.T) {
 			"<dd><p>Definition b</p></dd>\n" +
 			"</dl>\n" +
 			"\n<p>Text 2</p>\n",
+
+		"Term 1\n:   Definition a\n\n    Text 1\n\n    1. First\n    2. Second",
+		"<dl>\n" +
+			"<dt>Term 1</dt>\n" +
+			"<dd><p>Definition a</p>\n\n" +
+			"<p>Text 1</p>\n\n" +
+			"<ol>\n<li>First</li>\n<li>Second</li>\n</ol></dd>\n" +
+			"</dl>\n",
 	}
 	doTestsBlock(t, tests, EXTENSION_DEFINITION_LISTS)
 }
@ -985,6 +1053,9 @@ func TestFencedCodeBlock(t *testing.T) {
 		"``` go\nfunc foo() bool {\n\treturn true;\n}\n```\n",
 		"<pre><code class=\"language-go\">func foo() bool {\n\treturn true;\n}\n</code></pre>\n",

+		"``` go foo bar\nfunc foo() bool {\n\treturn true;\n}\n```\n",
+		"<pre><code class=\"language-go\">func foo() bool {\n\treturn true;\n}\n</code></pre>\n",
+
 		"``` c\n/* special & char < > \" escaping */\n```\n",
 		"<pre><code class=\"language-c\">/* special &amp; char &lt; &gt; &quot; escaping */\n</code></pre>\n",

@ -1025,7 +1096,7 @@ func TestFencedCodeBlock(t *testing.T) {
 		"<p>``` lisp\nno ending</p>\n",

 		"~~~ lisp\nend with language\n~~~ lisp\n",
-		"<p>~~~ lisp\nend with language\n~~~ lisp</p>\n",
+		"<pre><code class=\"language-lisp\">end with language\n</code></pre>\n\n<p>lisp</p>\n",

 		"```\nmismatched begin and end\n~~~\n",
 		"<p>```\nmismatched begin and end\n~~~</p>\n",
@ -1062,6 +1133,27 @@ func TestFencedCodeBlock(t *testing.T) {

 		"Some text before a fenced code block\n``` oz\ncode blocks breakup paragraphs\n```\nSome text in between\n``` oz\nmultiple code blocks work okay\n```\nAnd some text after a fenced code block",
 		"<p>Some text before a fenced code block</p>\n\n<pre><code class=\"language-oz\">code blocks breakup paragraphs\n</code></pre>\n\n<p>Some text in between</p>\n\n<pre><code class=\"language-oz\">multiple code blocks work okay\n</code></pre>\n\n<p>And some text after a fenced code block</p>\n",
+
+		"```\n[]:()\n```\n",
+		"<pre><code>[]:()\n</code></pre>\n",
+
+		"```\n[]:()\n[]:)\n[]:(\n[]:x\n[]:testing\n[:testing\n\n[]:\nlinebreak\n[]()\n\n[]:\n[]()\n```",
+		"<pre><code>[]:()\n[]:)\n[]:(\n[]:x\n[]:testing\n[:testing\n\n[]:\nlinebreak\n[]()\n\n[]:\n[]()\n</code></pre>\n",
+
+		"- test\n\n```\n  codeblock\n  ```\ntest\n",
+		"<ul>\n<li><p>test</p>\n\n<pre><code>codeblock\n</code></pre></li>\n</ul>\n\n<p>test</p>\n",
+
+		"- ```\n  codeblock\n  ```\n\n- test\n",
+		"<ul>\n<li><pre><code>codeblock\n</code></pre></li>\n\n<li><p>test</p></li>\n</ul>\n",
+
+		"- test\n- ```\n  codeblock\n  ```\n",
+		"<ul>\n<li>test</li>\n\n<li><pre><code>codeblock\n</code></pre></li>\n</ul>\n",
+
+		"- test\n```\ncodeblock\n```\n\n- test\n",
+		"<ul>\n<li><p>test</p>\n\n<pre><code>codeblock\n</code></pre></li>\n\n<li><p>test</p></li>\n</ul>\n",
+
+		"- test\n```go\nfunc foo() bool {\n\treturn true;\n}\n```\n\n- test\n",
+		"<ul>\n<li><p>test</p>\n\n<pre><code class=\"language-go\">func foo() bool {\n\treturn true;\n}\n</code></pre></li>\n\n<li><p>test</p></li>\n</ul>\n",
 	}
 	doTestsBlock(t, tests, EXTENSION_FENCED_CODE)
 }
@ -1437,6 +1529,9 @@ func TestFencedCodeBlock_EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK(t *testing.T) {
 		"``` go\nfunc foo() bool {\n\treturn true;\n}\n```\n",
 		"<pre><code class=\"language-go\">func foo() bool {\n\treturn true;\n}\n</code></pre>\n",

+		"``` go foo bar\nfunc foo() bool {\n\treturn true;\n}\n```\n",
+		"<pre><code class=\"language-go\">func foo() bool {\n\treturn true;\n}\n</code></pre>\n",
+
 		"``` c\n/* special & char < > \" escaping */\n```\n",
 		"<pre><code class=\"language-c\">/* special &amp; char &lt; &gt; &quot; escaping */\n</code></pre>\n",

@ -1477,7 +1572,7 @@ func TestFencedCodeBlock_EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK(t *testing.T) {
 		"<p>``` lisp\nno ending</p>\n",

 		"~~~ lisp\nend with language\n~~~ lisp\n",
-		"<p>~~~ lisp\nend with language\n~~~ lisp</p>\n",
+		"<pre><code class=\"language-lisp\">end with language\n</code></pre>\n\n<p>lisp</p>\n",

 		"```\nmismatched begin and end\n~~~\n",
 		"<p>```\nmismatched begin and end\n~~~</p>\n",
@ -1503,6 +1598,44 @@ func TestFencedCodeBlock_EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK(t *testing.T) {
 	doTestsBlock(t, tests, EXTENSION_FENCED_CODE|EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK)
 }

+func TestListWithFencedCodeBlock(t *testing.T) {
+	var tests = []string{
+		"1. one\n\n    ```\n    code\n    ```\n\n2. two\n",
+		"<ol>\n<li><p>one</p>\n\n<pre><code>code\n</code></pre></li>\n\n<li><p>two</p></li>\n</ol>\n",
+		// https://github.com/russross/blackfriday/issues/239
+		"1. one\n\n    ```\n    - code\n    ```\n\n2. two\n",
+		"<ol>\n<li><p>one</p>\n\n<pre><code>- code\n</code></pre></li>\n\n<li><p>two</p></li>\n</ol>\n",
+	}
+	doTestsBlock(t, tests, EXTENSION_FENCED_CODE)
+}
+
+func TestListWithMalformedFencedCodeBlock(t *testing.T) {
+	// Ensure that in the case of an unclosed fenced code block in a list,
+	// no source gets ommitted (even if it is malformed).
+	// See russross/blackfriday#372 for context.
+	var tests = []string{
+		"1. one\n\n    ```\n    code\n\n2. two\n",
+		"<ol>\n<li>one\n\n```\ncode\n\n2. two</li>\n</ol>\n",
+
+		"1. one\n\n    ```\n    - code\n\n2. two\n",
+		"<ol>\n<li>one\n\n```\n- code\n\n2. two</li>\n</ol>\n",
+	}
+	doTestsBlock(t, tests, EXTENSION_FENCED_CODE)
+}
+
+func TestListWithFencedCodeBlockNoExtensions(t *testing.T) {
+	// If there is a fenced code block in a list, and FencedCode is not set,
+	// lists should be processed normally.
+	var tests = []string{
+		"1. one\n\n    ```\n    code\n    ```\n\n2. two\n",
+		"<ol>\n<li><p>one</p>\n\n<p><code>\ncode\n</code></p></li>\n\n<li><p>two</p></li>\n</ol>\n",
+
+		"1. one\n\n    ```\n    - code\n    ```\n\n2. two\n",
+		"<ol>\n<li><p>one</p>\n\n<p>```</p>\n\n<ul>\n<li>code\n```</li>\n</ul></li>\n\n<li><p>two</p></li>\n</ol>\n",
+	}
+	doTestsBlock(t, tests, 0)
+}
+
 func TestTitleBlock_EXTENSION_TITLEBLOCK(t *testing.T) {
 	var tests = []string{
 		"% Some title\n" +
@ -1530,3 +1663,194 @@ func TestBlockComments(t *testing.T) {
 	}
 	doTestsBlock(t, tests, 0)
 }
+
+func TestCDATA(t *testing.T) {
+	var tests = []string{
+		"Some text\n\n<![CDATA[foo]]>\n",
+		"<p>Some text</p>\n\n<![CDATA[foo]]>\n",
+
+		"CDATA ]]\n\n<![CDATA[]]]]>\n",
+		"<p>CDATA ]]</p>\n\n<![CDATA[]]]]>\n",
+
+		"CDATA >\n\n<![CDATA[>]]>\n",
+		"<p>CDATA &gt;</p>\n\n<![CDATA[>]]>\n",
+
+		"Lots of text\n\n<![CDATA[lots of te><t\non\nseveral\nlines]]>\n",
+		"<p>Lots of text</p>\n\n<![CDATA[lots of te><t\non\nseveral\nlines]]>\n",
+
+		"<![CDATA[>]]>\n",
+		"<![CDATA[>]]>\n",
+	}
+	doTestsBlock(t, tests, 0)
+	doTestsBlock(t, []string{
+		"``` html\n<![CDATA[foo]]>\n```\n",
+		"<pre><code class=\"language-html\">&lt;![CDATA[foo]]&gt;\n</code></pre>\n",
+
+		"<![CDATA[\n``` python\ndef func():\n    pass\n```\n]]>\n",
+		"<![CDATA[\n``` python\ndef func():\n    pass\n```\n]]>\n",
+
+		`<![CDATA[
+> def func():
+>     pass
+]]>
+`,
+		`<![CDATA[
+> def func():
+>     pass
+]]>
+`,
+	}, EXTENSION_FENCED_CODE)
+}
+
+func TestIsFenceLine(t *testing.T) {
+	tests := []struct {
+		data            []byte
+		infoRequested   bool
+		newlineOptional bool
+		wantEnd         int
+		wantMarker      string
+		wantInfo        string
+	}{
+		{
+			data:    []byte("```"),
+			wantEnd: 0,
+		},
+		{
+			data:       []byte("```\nstuff here\n"),
+			wantEnd:    4,
+			wantMarker: "```",
+		},
+		{
+			data:          []byte("```\nstuff here\n"),
+			infoRequested: true,
+			wantEnd:       4,
+			wantMarker:    "```",
+		},
+		{
+			data:    []byte("stuff here\n```\n"),
+			wantEnd: 0,
+		},
+		{
+			data:            []byte("```"),
+			newlineOptional: true,
+			wantEnd:         3,
+			wantMarker:      "```",
+		},
+		{
+			data:            []byte("```"),
+			infoRequested:   true,
+			newlineOptional: true,
+			wantEnd:         3,
+			wantMarker:      "```",
+		},
+		{
+			data:            []byte("``` go"),
+			infoRequested:   true,
+			newlineOptional: true,
+			wantEnd:         6,
+			wantMarker:      "```",
+			wantInfo:        "go",
+		},
+		{
+			data:            []byte("``` go foo bar"),
+			infoRequested:   true,
+			newlineOptional: true,
+			wantEnd:         14,
+			wantMarker:      "```",
+			wantInfo:        "go foo bar",
+		},
+		{
+			data:            []byte("``` go foo bar  "),
+			infoRequested:   true,
+			newlineOptional: true,
+			wantEnd:         16,
+			wantMarker:      "```",
+			wantInfo:        "go foo bar",
+		},
+	}
+
+	for _, test := range tests {
+		var info *string
+		if test.infoRequested {
+			info = new(string)
+		}
+		end, marker := isFenceLine(test.data, info, "```", test.newlineOptional)
+		if got, want := end, test.wantEnd; got != want {
+			t.Errorf("got end %v, want %v", got, want)
+		}
+		if got, want := marker, test.wantMarker; got != want {
+			t.Errorf("got marker %q, want %q", got, want)
+		}
+		if test.infoRequested {
+			if got, want := *info, test.wantInfo; got != want {
+				t.Errorf("got info %q, want %q", got, want)
+			}
+		}
+	}
+}
+
+func TestJoinLines(t *testing.T) {
+	input := `# 标题
+
+第一
+行文字。
+
+第
+二
+行文字。
+`
+	result := `<h1>标题</h1>
+
+<p>第一行文字。</p>
+
+<p>第二行文字。</p>
+`
+	opt := Options{Extensions: commonExtensions | EXTENSION_JOIN_LINES}
+	renderer := HtmlRenderer(commonHtmlFlags, "", "")
+	output := MarkdownOptions([]byte(input), renderer, opt)
+
+	if string(output) != result {
+		t.Error("output dose not match.")
+	}
+}
+
+func TestSanitizedAnchorName(t *testing.T) {
+	tests := []struct {
+		text string
+		want string
+	}{
+		{
+			text: "This is a header",
+			want: "this-is-a-header",
+		},
+		{
+			text: "This is also          a header",
+			want: "this-is-also-a-header",
+		},
+		{
+			text: "main.go",
+			want: "main-go",
+		},
+		{
+			text: "Article 123",
+			want: "article-123",
+		},
+		{
+			text: "<- Let's try this, shall we?",
+			want: "let-s-try-this-shall-we",
+		},
+		{
+			text: "        ",
+			want: "",
+		},
+		{
+			text: "Hello, 世界",
+			want: "hello-世界",
+		},
+	}
+	for _, test := range tests {
+		if got := SanitizedAnchorName(test.text); got != test.want {
+			t.Errorf("SanitizedAnchorName(%q):\ngot %q\nwant %q", test.text, got, test.want)
+		}
+	}
+}
--- a/doc.go
+++ b/doc.go
@ -0,0 +1,32 @@
+// Package blackfriday is a Markdown processor.
+//
+// It translates plain text with simple formatting rules into HTML or LaTeX.
+//
+// Sanitized Anchor Names
+//
+// Blackfriday includes an algorithm for creating sanitized anchor names
+// corresponding to a given input text. This algorithm is used to create
+// anchors for headings when EXTENSION_AUTO_HEADER_IDS is enabled. The
+// algorithm is specified below, so that other packages can create
+// compatible anchor names and links to those anchors.
+//
+// The algorithm iterates over the input text, interpreted as UTF-8,
+// one Unicode code point (rune) at a time. All runes that are letters (category L)
+// or numbers (category N) are considered valid characters. They are mapped to
+// lower case, and included in the output. All other runes are considered
+// invalid characters. Invalid characters that preceed the first valid character,
+// as well as invalid character that follow the last valid character
+// are dropped completely. All other sequences of invalid characters
+// between two valid characters are replaced with a single dash character '-'.
+//
+// SanitizedAnchorName exposes this functionality, and can be used to
+// create compatible links to the anchor names generated by blackfriday.
+// This algorithm is also implemented in a small standalone package at
+// github.com/shurcooL/sanitized_anchor_name. It can be useful for clients
+// that want a small package and don't need full functionality of blackfriday.
+package blackfriday
+
+// NOTE: Keep Sanitized Anchor Name algorithm in sync with package
+//       github.com/shurcooL/sanitized_anchor_name.
+//       Otherwise, users of sanitized_anchor_name will get anchor names
+//       that are incompatible with those generated by blackfriday.
--- a/go.mod
+++ b/go.mod
@ -0,0 +1,3 @@
+module github.com/russross/blackfriday
+
+go 1.13
--- a/html.go
+++ b/html.go
@ -32,6 +32,7 @@ const (
 	HTML_SAFELINK                              // only link to trusted protocols
 	HTML_NOFOLLOW_LINKS                        // only link with rel="nofollow"
 	HTML_NOREFERRER_LINKS                      // only link with rel="noreferrer"
+	HTML_NOOPENER_LINKS                        // only link with rel="noopener"
 	HTML_HREF_TARGET_BLANK                     // add a blank target
 	HTML_TOC                                   // generate a table of contents
 	HTML_OMIT_CONTENTS                         // skip the main contents (for a standalone table of contents)
@ -42,6 +43,7 @@ const (
 	HTML_SMARTYPANTS_DASHES                    // enable smart dashes (with HTML_USE_SMARTYPANTS)
 	HTML_SMARTYPANTS_LATEX_DASHES              // enable LaTeX-style dashes (with HTML_USE_SMARTYPANTS and HTML_SMARTYPANTS_DASHES)
 	HTML_SMARTYPANTS_ANGLED_QUOTES             // enable angled double quotes (with HTML_USE_SMARTYPANTS) for double quotes rendering
+	HTML_SMARTYPANTS_QUOTES_NBSP               // enable "French guillemets" (with HTML_USE_SMARTYPANTS)
 	HTML_FOOTNOTE_RETURN_LINKS                 // generate a link at the end of a footnote to return to the source
 )

@ -254,33 +256,21 @@ func (options *Html) HRule(out *bytes.Buffer) {
 	out.WriteByte('\n')
 }

-func (options *Html) BlockCode(out *bytes.Buffer, text []byte, lang string) {
+func (options *Html) BlockCode(out *bytes.Buffer, text []byte, info string) {
 	doubleSpace(out)

-	// parse out the language names/classes
-	count := 0
-	for _, elt := range strings.Fields(lang) {
-		if elt[0] == '.' {
-			elt = elt[1:]
-		}
-		if len(elt) == 0 {
-			continue
-		}
-		if count == 0 {
-			out.WriteString("<pre><code class=\"language-")
-		} else {
-			out.WriteByte(' ')
-		}
-		attrEscape(out, []byte(elt))
-		count++
+	endOfLang := strings.IndexAny(info, "\t ")
+	if endOfLang < 0 {
+		endOfLang = len(info)
 	}
-
-	if count == 0 {
+	lang := info[:endOfLang]
+	if len(lang) == 0 || lang == "." {
 		out.WriteString("<pre><code>")
 	} else {
+		out.WriteString("<pre><code class=\"language-")
+		attrEscape(out, []byte(lang))
 		out.WriteString("\">")
 	}
-
 	attrEscape(out, text)
 	out.WriteString("</code></pre>\n")
 }
@ -456,6 +446,9 @@ func (options *Html) AutoLink(out *bytes.Buffer, link []byte, kind int) {
 	if options.flags&HTML_NOREFERRER_LINKS != 0 && !isRelativeLink(link) {
 		relAttrs = append(relAttrs, "noreferrer")
 	}
+	if options.flags&HTML_NOOPENER_LINKS != 0 && !isRelativeLink(link) {
+		relAttrs = append(relAttrs, "noopener")
+	}
 	if len(relAttrs) > 0 {
 		out.WriteString(fmt.Sprintf("\" rel=\"%s", strings.Join(relAttrs, " ")))
 	}
@ -570,6 +563,9 @@ func (options *Html) Link(out *bytes.Buffer, link []byte, title []byte, content
 	if options.flags&HTML_NOREFERRER_LINKS != 0 && !isRelativeLink(link) {
 		relAttrs = append(relAttrs, "noreferrer")
 	}
+	if options.flags&HTML_NOOPENER_LINKS != 0 && !isRelativeLink(link) {
+		relAttrs = append(relAttrs, "noopener")
+	}
 	if len(relAttrs) > 0 {
 		out.WriteString(fmt.Sprintf("\" rel=\"%s", strings.Join(relAttrs, " ")))
 	}
@ -619,7 +615,7 @@ func (options *Html) FootnoteRef(out *bytes.Buffer, ref []byte, id int) {
 	out.WriteString(`fnref:`)
 	out.WriteString(options.parameters.FootnoteAnchorPrefix)
 	out.Write(slug)
-	out.WriteString(`"><a rel="footnote" href="#`)
+	out.WriteString(`"><a href="#`)
 	out.WriteString(`fn:`)
 	out.WriteString(options.parameters.FootnoteAnchorPrefix)
 	out.Write(slug)
--- a/inline.go
+++ b/inline.go
@ -170,6 +170,10 @@ func lineBreak(p *parser, out *bytes.Buffer, data []byte, offset int) int {
 	precededByBackslash := offset >= 1 && data[offset-1] == '\\' // see http://spec.commonmark.org/0.18/#example-527
 	precededByBackslash = precededByBackslash && p.flags&EXTENSION_BACKSLASH_LINE_BREAK != 0

+	if p.flags&EXTENSION_JOIN_LINES != 0 {
+		return 1
+	}
+
 	// should there be a hard line break here?
 	if p.flags&EXTENSION_HARD_LINE_BREAK == 0 && !precededByTwoSpaces && !precededByBackslash {
 		return 0
@ -240,13 +244,15 @@ func link(p *parser, out *bytes.Buffer, data []byte, offset int) int {
 		i++
 	}

+	brace := 0
+
 	// look for the matching closing bracket
 	for level := 1; level > 0 && i < len(data); i++ {
 		switch {
 		case data[i] == '\n':
 			textHasNl = true

-		case data[i-1] == '\\':
+		case isBackslashEscaped(data, i):
 			continue

 		case data[i] == '[':
@ -273,8 +279,8 @@ func link(p *parser, out *bytes.Buffer, data []byte, offset int) int {
 		i++
 	}

-	// inline style link
 	switch {
+	// inline style link
 	case i < len(data) && data[i] == '(':
 		// skip initial whitespace
 		i++
@ -285,14 +291,27 @@ func link(p *parser, out *bytes.Buffer, data []byte, offset int) int {

 		linkB := i

-		// look for link end: ' " )
+		// look for link end: ' " ), check for new opening braces and take this
+		// into account, this may lead for overshooting and probably will require
+		// some fine-tuning.
 	findlinkend:
 		for i < len(data) {
 			switch {
 			case data[i] == '\\':
 				i += 2

-			case data[i] == ')' || data[i] == '\'' || data[i] == '"':
+			case data[i] == '(':
+				brace++
+				i++
+
+			case data[i] == ')':
+				if brace <= 0 {
+					break findlinkend
+				}
+				brace--
+				i++
+
+			case data[i] == '\'' || data[i] == '"':
 				break findlinkend

 			default:
@ -469,6 +488,7 @@ func link(p *parser, out *bytes.Buffer, data []byte, offset int) int {
 			}

 			p.notes = append(p.notes, ref)
+			p.notesRecord[string(ref.link)] = struct{}{}

 			link = ref.link
 			title = ref.title
@ -479,9 +499,10 @@ func link(p *parser, out *bytes.Buffer, data []byte, offset int) int {
 				return 0
 			}

-			if t == linkDeferredFootnote {
+			if t == linkDeferredFootnote && !p.isFootnote(lr) {
 				lr.noteId = len(p.notes) + 1
 				p.notes = append(p.notes, lr)
+				p.notesRecord[string(lr.link)] = struct{}{}
 			}

 			// keep link and title from reference
@ -560,7 +581,7 @@ func link(p *parser, out *bytes.Buffer, data []byte, offset int) int {
 	return i
 }

-func (p *parser) inlineHtmlComment(out *bytes.Buffer, data []byte) int {
+func (p *parser) inlineHTMLComment(out *bytes.Buffer, data []byte) int {
 	if len(data) < 5 {
 		return 0
 	}
@ -584,7 +605,7 @@ func leftAngle(p *parser, out *bytes.Buffer, data []byte, offset int) int {
 	data = data[offset:]
 	altype := LINK_TYPE_NOT_AUTOLINK
 	end := tagLength(data, &altype)
-	if size := p.inlineHtmlComment(out, data); size > 0 {
+	if size := p.inlineHTMLComment(out, data); size > 0 {
 		end = size
 	}
 	if end > 2 {
@ -923,7 +944,7 @@ func isMailtoAutoLink(data []byte) int {
 			nb++

 		case '-', '.', '_':
-			break
+			// Do nothing.

 		case '>':
 			if nb == 1 {
--- a/inline_test.go
+++ b/inline_test.go
@ -15,9 +15,8 @@ package blackfriday

 import (
 	"regexp"
-	"testing"
-
 	"strings"
+	"testing"
 )

 func runMarkdownInline(input string, opts Options, htmlFlags int, params HtmlRendererParameters) string {
@ -60,13 +59,11 @@ func doTestsInlineParam(t *testing.T, tests []string, opts Options, htmlFlags in
 	params HtmlRendererParameters) {
 	// catch and report panics
 	var candidate string
-	/*
-		defer func() {
-			if err := recover(); err != nil {
-				t.Errorf("\npanic while processing [%#v] (%v)\n", candidate, err)
-			}
-		}()
-	*/
+	defer func() {
+		if err := recover(); err != nil {
+			t.Errorf("\npanic while processing [%#v]: %s\n", candidate, err)
+		}
+	}()

 	for i := 0; i+1 < len(tests); i += 2 {
 		input := tests[i]
@ -156,10 +153,35 @@ func TestEmphasis(t *testing.T) {

 		"*What is A\\* algorithm?*\n",
 		"<p><em>What is A* algorithm?</em></p>\n",
+
+		"some para_graph with _emphasised_ text.\n",
+		"<p>some para_graph with <em>emphasised</em> text.</p>\n",
+
+		"some paragraph with _emphasised_ te_xt.\n",
+		"<p>some paragraph with <em>emphasised</em> te_xt.</p>\n",
+
+		"some paragraph with t_wo bi_ts of _emphasised_ text.\n",
+		"<p>some paragraph with t<em>wo bi</em>ts of <em>emphasised</em> text.</p>\n",
+
+		"un*frigging*believable\n",
+		"<p>un<em>frigging</em>believable</p>\n",
 	}
 	doTestsInline(t, tests)
 }

+func TestNoIntraEmphasis(t *testing.T) {
+	tests := []string{
+		"some para_graph with _emphasised_ text.\n",
+		"<p>some para_graph with <em>emphasised</em> text.</p>\n",
+
+		"un*frigging*believable\n",
+		"<p>un*frigging*believable</p>\n",
+	}
+	doTestsInlineParam(t, tests, Options{
+		Extensions: EXTENSION_NO_INTRA_EMPHASIS},
+		0, HtmlRendererParameters{})
+}
+
 func TestReferenceOverride(t *testing.T) {
 	var tests = []string{
 		"test [ref1][]\n",
@ -541,6 +563,20 @@ func TestInlineLink(t *testing.T) {

 		"[link](<../>)\n",
 		"<p><a href=\"../\">link</a></p>\n",
+
+		// Issue 116 in blackfriday
+		"![](http://www.broadgate.co.uk/Content/Upload/DetailImages/Cyclus700(1).jpg)",
+		"<p><img src=\"http://www.broadgate.co.uk/Content/Upload/DetailImages/Cyclus700(1).jpg\" alt=\"\" /></p>\n",
+
+		// no closing ), autolinking detects the url next
+		"[disambiguation](http://en.wikipedia.org/wiki/Disambiguation_(disambiguation) is the",
+		"<p>[disambiguation](<a href=\"http://en.wikipedia.org/wiki/Disambiguation_(disambiguation\">http://en.wikipedia.org/wiki/Disambiguation_(disambiguation</a>) is the</p>\n",
+
+		"[disambiguation](http://en.wikipedia.org/wiki/Disambiguation_(disambiguation)) is the",
+		"<p><a href=\"http://en.wikipedia.org/wiki/Disambiguation_(disambiguation)\">disambiguation</a> is the</p>\n",
+
+		"[disambiguation](http://en.wikipedia.org/wiki/Disambiguation_(disambiguation))",
+		"<p><a href=\"http://en.wikipedia.org/wiki/Disambiguation_(disambiguation)\">disambiguation</a></p>\n",
 	}
 	doLinkTestsInline(t, tests)

@ -588,6 +624,25 @@ func TestRelAttrLink(t *testing.T) {
 	}
 	doTestsInlineParam(t, nofollownoreferrerTests, Options{}, HTML_SAFELINK|HTML_NOFOLLOW_LINKS|HTML_NOREFERRER_LINKS,
 		HtmlRendererParameters{})
+
+	var noopenerTests = []string{
+		"[foo](http://bar.com/foo/)\n",
+		"<p><a href=\"http://bar.com/foo/\" rel=\"noopener\">foo</a></p>\n",
+
+		"[foo](/bar/)\n",
+		"<p><a href=\"/bar/\">foo</a></p>\n",
+	}
+	doTestsInlineParam(t, noopenerTests, Options{}, HTML_SAFELINK|HTML_NOOPENER_LINKS, HtmlRendererParameters{})
+
+	var nofollownoreferrernoopenerTests = []string{
+		"[foo](http://bar.com/foo/)\n",
+		"<p><a href=\"http://bar.com/foo/\" rel=\"nofollow noreferrer noopener\">foo</a></p>\n",
+
+		"[foo](/bar/)\n",
+		"<p><a href=\"/bar/\">foo</a></p>\n",
+	}
+	doTestsInlineParam(t, nofollownoreferrernoopenerTests, Options{},
+		HTML_SAFELINK|HTML_NOOPENER_LINKS|HTML_NOFOLLOW_LINKS|HTML_NOREFERRER_LINKS, HtmlRendererParameters{})
 }

 func TestHrefTargetBlank(t *testing.T) {
@ -681,6 +736,9 @@ func TestReferenceLink(t *testing.T) {

 		"[ref]\n   [ref]: ../url/ \"title\"\n",
 		"<p><a href=\"../url/\" title=\"title\">ref</a></p>\n",
+
+		"[link][ref]\n   [ref]: /url/",
+		"<p><a href=\"/url/\">link</a></p>\n",
 	}
 	doLinkTestsInline(t, tests)
 }
@ -814,7 +872,7 @@ func TestAutoLink(t *testing.T) {

 var footnoteTests = []string{
 	"testing footnotes.[^a]\n\n[^a]: This is the note\n",
-	`<p>testing footnotes.<sup class="footnote-ref" id="fnref:a"><a rel="footnote" href="#fn:a">1</a></sup></p>
+	`<p>testing footnotes.<sup class="footnote-ref" id="fnref:a"><a href="#fn:a">1</a></sup></p>
 <div class="footnotes">

 <hr />
@ -838,7 +896,7 @@ var footnoteTests = []string{

 No longer in the footnote
 `,
-	`<p>testing long<sup class="footnote-ref" id="fnref:b"><a rel="footnote" href="#fn:b">1</a></sup> notes.</p>
+	`<p>testing long<sup class="footnote-ref" id="fnref:b"><a href="#fn:b">1</a></sup> notes.</p>

 <p>No longer in the footnote</p>
 <div class="footnotes">
@ -874,7 +932,7 @@ what happens here
 [note]: /link/c

 `,
-	`<p>testing<sup class="footnote-ref" id="fnref:c"><a rel="footnote" href="#fn:c">1</a></sup> multiple<sup class="footnote-ref" id="fnref:d"><a rel="footnote" href="#fn:d">2</a></sup> notes.</p>
+	`<p>testing<sup class="footnote-ref" id="fnref:c"><a href="#fn:c">1</a></sup> multiple<sup class="footnote-ref" id="fnref:d"><a href="#fn:d">2</a></sup> notes.</p>

 <p>omg</p>

@ -893,7 +951,7 @@ what happens here
 `,

 	"testing inline^[this is the note] notes.\n",
-	`<p>testing inline<sup class="footnote-ref" id="fnref:this-is-the-note"><a rel="footnote" href="#fn:this-is-the-note">1</a></sup> notes.</p>
+	`<p>testing inline<sup class="footnote-ref" id="fnref:this-is-the-note"><a href="#fn:this-is-the-note">1</a></sup> notes.</p>
 <div class="footnotes">

 <hr />
@ -905,7 +963,7 @@ what happens here
 `,

 	"testing multiple[^1] types^[inline note] of notes[^2]\n\n[^2]: the second deferred note\n[^1]: the first deferred note\n\n\twhich happens to be a block\n",
-	`<p>testing multiple<sup class="footnote-ref" id="fnref:1"><a rel="footnote" href="#fn:1">1</a></sup> types<sup class="footnote-ref" id="fnref:inline-note"><a rel="footnote" href="#fn:inline-note">2</a></sup> of notes<sup class="footnote-ref" id="fnref:2"><a rel="footnote" href="#fn:2">3</a></sup></p>
+	`<p>testing multiple<sup class="footnote-ref" id="fnref:1"><a href="#fn:1">1</a></sup> types<sup class="footnote-ref" id="fnref:inline-note"><a href="#fn:inline-note">2</a></sup> of notes<sup class="footnote-ref" id="fnref:2"><a href="#fn:2">3</a></sup></p>
 <div class="footnotes">

 <hr />
@ -928,7 +986,7 @@ what happens here

    may be multiple paragraphs.
 `,
-	`<p>This is a footnote<sup class="footnote-ref" id="fnref:1"><a rel="footnote" href="#fn:1">1</a></sup><sup class="footnote-ref" id="fnref:and-this-is-an-i"><a rel="footnote" href="#fn:and-this-is-an-i">2</a></sup></p>
+	`<p>This is a footnote<sup class="footnote-ref" id="fnref:1"><a href="#fn:1">1</a></sup><sup class="footnote-ref" id="fnref:and-this-is-an-i"><a href="#fn:and-this-is-an-i">2</a></sup></p>
 <div class="footnotes">

 <hr />
@ -944,13 +1002,13 @@ what happens here
 `,

 	"empty footnote[^]\n\n[^]: fn text",
-	"<p>empty footnote<sup class=\"footnote-ref\" id=\"fnref:\"><a rel=\"footnote\" href=\"#fn:\">1</a></sup></p>\n<div class=\"footnotes\">\n\n<hr />\n\n<ol>\n<li id=\"fn:\">fn text\n</li>\n</ol>\n</div>\n",
+	"<p>empty footnote<sup class=\"footnote-ref\" id=\"fnref:\"><a href=\"#fn:\">1</a></sup></p>\n<div class=\"footnotes\">\n\n<hr />\n\n<ol>\n<li id=\"fn:\">fn text\n</li>\n</ol>\n</div>\n",

 	"Some text.[^note1]\n\n[^note1]: fn1",
-	"<p>Some text.<sup class=\"footnote-ref\" id=\"fnref:note1\"><a rel=\"footnote\" href=\"#fn:note1\">1</a></sup></p>\n<div class=\"footnotes\">\n\n<hr />\n\n<ol>\n<li id=\"fn:note1\">fn1\n</li>\n</ol>\n</div>\n",
+	"<p>Some text.<sup class=\"footnote-ref\" id=\"fnref:note1\"><a href=\"#fn:note1\">1</a></sup></p>\n<div class=\"footnotes\">\n\n<hr />\n\n<ol>\n<li id=\"fn:note1\">fn1\n</li>\n</ol>\n</div>\n",

 	"Some text.[^note1][^note2]\n\n[^note1]: fn1\n[^note2]: fn2\n",
-	"<p>Some text.<sup class=\"footnote-ref\" id=\"fnref:note1\"><a rel=\"footnote\" href=\"#fn:note1\">1</a></sup><sup class=\"footnote-ref\" id=\"fnref:note2\"><a rel=\"footnote\" href=\"#fn:note2\">2</a></sup></p>\n<div class=\"footnotes\">\n\n<hr />\n\n<ol>\n<li id=\"fn:note1\">fn1\n</li>\n<li id=\"fn:note2\">fn2\n</li>\n</ol>\n</div>\n",
+	"<p>Some text.<sup class=\"footnote-ref\" id=\"fnref:note1\"><a href=\"#fn:note1\">1</a></sup><sup class=\"footnote-ref\" id=\"fnref:note2\"><a href=\"#fn:note2\">2</a></sup></p>\n<div class=\"footnotes\">\n\n<hr />\n\n<ol>\n<li id=\"fn:note1\">fn1\n</li>\n<li id=\"fn:note2\">fn2\n</li>\n</ol>\n</div>\n",

 	`Bla bla [^1] [WWW][w3]

@ -958,7 +1016,7 @@ what happens here

 [w3]: http://www.w3.org/
 `,
-	`<p>Bla bla <sup class="footnote-ref" id="fnref:1"><a rel="footnote" href="#fn:1">1</a></sup> <a href="http://www.w3.org/">WWW</a></p>
+	`<p>Bla bla <sup class="footnote-ref" id="fnref:1"><a href="#fn:1">1</a></sup> <a href="http://www.w3.org/">WWW</a></p>
 <div class="footnotes">

 <hr />
@ -974,7 +1032,7 @@ what happens here

 [^fn1]: Fine print
 `,
-	`<p>This is exciting!<sup class="footnote-ref" id="fnref:fn1"><a rel="footnote" href="#fn:fn1">1</a></sup></p>
+	`<p>This is exciting!<sup class="footnote-ref" id="fnref:fn1"><a href="#fn:fn1">1</a></sup></p>
 <div class="footnotes">

 <hr />
@ -984,6 +1042,28 @@ what happens here
 </li>
 </ol>
 </div>
+`,
+	`testing footnotes.[^a]
+
+test footnotes the second.[^b]
+
+[^a]: This is the first note[^a].
+[^b]: this is the second note.[^a]
+`,
+	`<p>testing footnotes.<sup class="footnote-ref" id="fnref:a"><a href="#fn:a">1</a></sup></p>
+
+<p>test footnotes the second.<sup class="footnote-ref" id="fnref:b"><a href="#fn:b">2</a></sup></p>
+<div class="footnotes">
+
+<hr />
+
+<ol>
+<li id="fn:a">This is the first note<sup class="footnote-ref" id="fnref:a"><a href="#fn:a">1</a></sup>.
+</li>
+<li id="fn:b">this is the second note.<sup class="footnote-ref" id="fnref:a"><a href="#fn:a">1</a></sup>
+</li>
+</ol>
+</div>
 `,
 }

@ -1025,18 +1105,46 @@ func TestNestedFootnotes(t *testing.T) {

 [^fn2]:
  Obelisk`,
-		`<p>Paragraph.<sup class="footnote-ref" id="fnref:fn1"><a rel="footnote" href="#fn:fn1">1</a></sup></p>
+		`<p>Paragraph.<sup class="footnote-ref" id="fnref:fn1"><a href="#fn:fn1">1</a></sup></p>
 <div class="footnotes">

 <hr />

 <ol>
-<li id="fn:fn1">Asterisk<sup class="footnote-ref" id="fnref:fn2"><a rel="footnote" href="#fn:fn2">2</a></sup>
+<li id="fn:fn1">Asterisk<sup class="footnote-ref" id="fnref:fn2"><a href="#fn:fn2">2</a></sup>
 </li>
 <li id="fn:fn2">Obelisk
 </li>
 </ol>
 </div>
+`,
+		`This uses footnote A.[^A]
+
+This uses footnote C.[^C]
+
+[^A]:
+  A note. use itself.[^A]
+[^B]:
+  B note, uses A to test duplicate.[^A]
+[^C]:
+  C note, uses B.[^B]
+`,
+		`<p>This uses footnote A.<sup class="footnote-ref" id="fnref:A"><a href="#fn:A">1</a></sup></p>
+
+<p>This uses footnote C.<sup class="footnote-ref" id="fnref:C"><a href="#fn:C">2</a></sup></p>
+<div class="footnotes">
+
+<hr />
+
+<ol>
+<li id="fn:A">A note. use itself.<sup class="footnote-ref" id="fnref:A"><a href="#fn:A">1</a></sup>
+</li>
+<li id="fn:C">C note, uses B.<sup class="footnote-ref" id="fnref:B"><a href="#fn:B">3</a></sup>
+</li>
+<li id="fn:B">B note, uses A to test duplicate.<sup class="footnote-ref" id="fnref:A"><a href="#fn:A">1</a></sup>
+</li>
+</ol>
+</div>
 `,
 	}
 	doTestsInlineParam(t, tests, Options{Extensions: EXTENSION_FOOTNOTES}, 0,
@ -1084,6 +1192,18 @@ func TestSmartDoubleQuotes(t *testing.T) {
 	doTestsInlineParam(t, tests, Options{}, HTML_USE_SMARTYPANTS, HtmlRendererParameters{})
 }

+func TestSmartDoubleQuotesNbsp(t *testing.T) {
+	var tests = []string{
+		"this should be normal \"quoted\" text.\n",
+		"<p>this should be normal &ldquo;&nbsp;quoted&nbsp;&rdquo; text.</p>\n",
+		"this \" single double\n",
+		"<p>this &ldquo;&nbsp; single double</p>\n",
+		"two pair of \"some\" quoted \"text\".\n",
+		"<p>two pair of &ldquo;&nbsp;some&nbsp;&rdquo; quoted &ldquo;&nbsp;text&nbsp;&rdquo;.</p>\n"}
+
+	doTestsInlineParam(t, tests, Options{}, HTML_USE_SMARTYPANTS|HTML_SMARTYPANTS_QUOTES_NBSP, HtmlRendererParameters{})
+}
+
 func TestSmartAngledDoubleQuotes(t *testing.T) {
 	var tests = []string{
 		"this should be angled \"quoted\" text.\n",
@ -1096,6 +1216,18 @@ func TestSmartAngledDoubleQuotes(t *testing.T) {
 	doTestsInlineParam(t, tests, Options{}, HTML_USE_SMARTYPANTS|HTML_SMARTYPANTS_ANGLED_QUOTES, HtmlRendererParameters{})
 }

+func TestSmartAngledDoubleQuotesNbsp(t *testing.T) {
+	var tests = []string{
+		"this should be angled \"quoted\" text.\n",
+		"<p>this should be angled &laquo;&nbsp;quoted&nbsp;&raquo; text.</p>\n",
+		"this \" single double\n",
+		"<p>this &laquo;&nbsp; single double</p>\n",
+		"two pair of \"some\" quoted \"text\".\n",
+		"<p>two pair of &laquo;&nbsp;some&nbsp;&raquo; quoted &laquo;&nbsp;text&nbsp;&raquo;.</p>\n"}
+
+	doTestsInlineParam(t, tests, Options{}, HTML_USE_SMARTYPANTS|HTML_SMARTYPANTS_ANGLED_QUOTES|HTML_SMARTYPANTS_QUOTES_NBSP, HtmlRendererParameters{})
+}
+
 func TestSmartFractions(t *testing.T) {
 	var tests = []string{
 		"1/2, 1/4 and 3/4; 1/4th and 3/4ths\n",
@ -1151,3 +1283,9 @@ func TestDisableSmartDashes(t *testing.T) {
 		HTML_USE_SMARTYPANTS|HTML_SMARTYPANTS_LATEX_DASHES,
 		HtmlRendererParameters{})
 }
+
+func BenchmarkSmartDoubleQuotes(b *testing.B) {
+	for i := 0; i < b.N; i++ {
+		runMarkdownInline("this should be normal \"quoted\" text.\n", Options{}, HTML_USE_SMARTYPANTS, HtmlRendererParameters{})
+	}
+}
--- a/latex.go
+++ b/latex.go
@ -17,6 +17,7 @@ package blackfriday

 import (
 	"bytes"
+	"strings"
 )

 // Latex is a type that implements the Renderer interface for LaTeX output.
@ -39,16 +40,17 @@ func (options *Latex) GetFlags() int {
 }

 // render code chunks using verbatim, or listings if we have a language
-func (options *Latex) BlockCode(out *bytes.Buffer, text []byte, lang string) {
-	if lang == "" {
+func (options *Latex) BlockCode(out *bytes.Buffer, text []byte, info string) {
+	if info == "" {
 		out.WriteString("\n\\begin{verbatim}\n")
 	} else {
+		lang := strings.Fields(info)[0]
 		out.WriteString("\n\\begin{lstlisting}[language=")
 		out.WriteString(lang)
 		out.WriteString("]\n")
 	}
 	out.Write(text)
-	if lang == "" {
+	if info == "" {
 		out.WriteString("\n\\end{verbatim}\n")
 	} else {
 		out.WriteString("\n\\end{lstlisting}\n")
--- a/markdown.go
+++ b/markdown.go
@ -13,9 +13,6 @@
 //
 //

-// Blackfriday markdown processor.
-//
-// Translates plain text with simple formatting rules into HTML or LaTeX.
 package blackfriday

 import (
@ -25,7 +22,7 @@ import (
 	"unicode/utf8"
 )

-const VERSION = "1.4"
+const VERSION = "1.5"

 // These are the supported markdown parsing extensions.
 // OR these values together to select multiple extensions.
@ -46,6 +43,7 @@ const (
 	EXTENSION_AUTO_HEADER_IDS                        // Create the header ID from the text
 	EXTENSION_BACKSLASH_LINE_BREAK                   // translate trailing backslashes into line breaks
 	EXTENSION_DEFINITION_LISTS                       // render definition lists
+	EXTENSION_JOIN_LINES                             // delete newline and join lines

 	commonHtmlFlags = 0 |
 		HTML_USE_XHTML |
@ -105,46 +103,48 @@ const (
 // blockTags is a set of tags that are recognized as HTML block tags.
 // Any of these can be included in markdown text without special escaping.
 var blockTags = map[string]struct{}{
-	"blockquote": struct{}{},
-	"del":        struct{}{},
-	"div":        struct{}{},
-	"dl":         struct{}{},
-	"fieldset":   struct{}{},
-	"form":       struct{}{},
-	"h1":         struct{}{},
-	"h2":         struct{}{},
-	"h3":         struct{}{},
-	"h4":         struct{}{},
-	"h5":         struct{}{},
-	"h6":         struct{}{},
-	"iframe":     struct{}{},
-	"ins":        struct{}{},
-	"math":       struct{}{},
-	"noscript":   struct{}{},
-	"ol":         struct{}{},
-	"pre":        struct{}{},
-	"p":          struct{}{},
-	"script":     struct{}{},
-	"style":      struct{}{},
-	"table":      struct{}{},
-	"ul":         struct{}{},
+	"blockquote": {},
+	"del":        {},
+	"div":        {},
+	"dl":         {},
+	"fieldset":   {},
+	"form":       {},
+	"h1":         {},
+	"h2":         {},
+	"h3":         {},
+	"h4":         {},
+	"h5":         {},
+	"h6":         {},
+	"iframe":     {},
+	"ins":        {},
+	"math":       {},
+	"noscript":   {},
+	"ol":         {},
+	"pre":        {},
+	"p":          {},
+	"script":     {},
+	"style":      {},
+	"table":      {},
+	"ul":         {},

 	// HTML5
-	"address":    struct{}{},
-	"article":    struct{}{},
-	"aside":      struct{}{},
-	"canvas":     struct{}{},
-	"figcaption": struct{}{},
-	"figure":     struct{}{},
-	"footer":     struct{}{},
-	"header":     struct{}{},
-	"hgroup":     struct{}{},
-	"main":       struct{}{},
-	"nav":        struct{}{},
-	"output":     struct{}{},
-	"progress":   struct{}{},
-	"section":    struct{}{},
-	"video":      struct{}{},
+	"address":    {},
+	"article":    {},
+	"aside":      {},
+	"canvas":     {},
+	"details":    {},
+	"figcaption": {},
+	"figure":     {},
+	"footer":     {},
+	"header":     {},
+	"hgroup":     {},
+	"main":       {},
+	"nav":        {},
+	"output":     {},
+	"progress":   {},
+	"section":    {},
+	"summary":    {},
+	"video":      {},
 }

 // Renderer is the rendering interface.
@ -161,7 +161,7 @@ var blockTags = map[string]struct{}{
 // Currently Html and Latex implementations are provided
 type Renderer interface {
 	// block-level callbacks
-	BlockCode(out *bytes.Buffer, text []byte, lang string)
+	BlockCode(out *bytes.Buffer, text []byte, infoString string)
 	BlockQuote(out *bytes.Buffer, text []byte)
 	BlockHtml(out *bytes.Buffer, text []byte)
 	Header(out *bytes.Buffer, text func() bool, level int, id string)
@ -220,7 +220,8 @@ type parser struct {
 	// Footnotes need to be ordered as well as available to quickly check for
 	// presence. If a ref is also a footnote, it's stored both in refs and here
 	// in notes. Slice is nil if footnotes not enabled.
-	notes []*reference
+	notes       []*reference
+	notesRecord map[string]struct{}
 }

 func (p *parser) getRef(refid string) (ref *reference, found bool) {
@ -243,6 +244,11 @@ func (p *parser) getRef(refid string) (ref *reference, found bool) {
 	return ref, found
 }

+func (p *parser) isFootnote(ref *reference) bool {
+	_, ok := p.notesRecord[string(ref.link)]
+	return ok
+}
+
 //
 //
 // Public interface
@ -378,6 +384,7 @@ func MarkdownOptions(input []byte, renderer Renderer, opts Options) []byte {

 	if extensions&EXTENSION_FOOTNOTES != 0 {
 		p.notes = make([]*reference, 0)
+		p.notesRecord = make(map[string]struct{})
 	}

 	first := firstPass(p, input)
@ -386,9 +393,9 @@ func MarkdownOptions(input []byte, renderer Renderer, opts Options) []byte {
 }

 // first pass:
-// - extract references
-// - expand tabs
 // - normalize newlines
+// - extract references (outside of fenced code blocks)
+// - expand tabs (outside of fenced code blocks)
 // - copy everything else
 func firstPass(p *parser, input []byte) []byte {
 	var out bytes.Buffer
@ -396,46 +403,46 @@ func firstPass(p *parser, input []byte) []byte {
 	if p.flags&EXTENSION_TAB_SIZE_EIGHT != 0 {
 		tabSize = TAB_SIZE_EIGHT
 	}
-	beg, end := 0, 0
+	beg := 0
 	lastFencedCodeBlockEnd := 0
-	for beg < len(input) { // iterate over lines
-		if end = isReference(p, input[beg:], tabSize); end > 0 {
-			beg += end
-		} else { // skip to the next line
-			end = beg
-			for end < len(input) && input[end] != '\n' && input[end] != '\r' {
-				end++
-			}
-
-			if p.flags&EXTENSION_FENCED_CODE != 0 {
-				// track fenced code block boundaries to suppress tab expansion
-				// inside them:
-				if beg >= lastFencedCodeBlockEnd {
-					if i := p.fencedCode(&out, input[beg:], false); i > 0 {
-						lastFencedCodeBlockEnd = beg + i
-					}
-				}
-			}
-
-			// add the line body if present
-			if end > beg {
-				if end < lastFencedCodeBlockEnd { // Do not expand tabs while inside fenced code blocks.
-					out.Write(input[beg:end])
-				} else {
-					expandTabs(&out, input[beg:end], tabSize)
-				}
-			}
-			out.WriteByte('\n')
-
-			if end < len(input) && input[end] == '\r' {
-				end++
-			}
-			if end < len(input) && input[end] == '\n' {
-				end++
-			}
-
-			beg = end
+	for beg < len(input) {
+		// Find end of this line, then process the line.
+		end := beg
+		for end < len(input) && input[end] != '\n' && input[end] != '\r' {
+			end++
 		}
+
+		if p.flags&EXTENSION_FENCED_CODE != 0 {
+			// track fenced code block boundaries to suppress tab expansion
+			// and reference extraction inside them:
+			if beg >= lastFencedCodeBlockEnd {
+				if i := p.fencedCodeBlock(&out, input[beg:], false); i > 0 {
+					lastFencedCodeBlockEnd = beg + i
+				}
+			}
+		}
+
+		// add the line body if present
+		if end > beg {
+			if end < lastFencedCodeBlockEnd { // Do not expand tabs while inside fenced code blocks.
+				out.Write(input[beg:end])
+			} else if refEnd := isReference(p, input[beg:], tabSize); refEnd > 0 {
+				beg += refEnd
+				continue
+			} else {
+				expandTabs(&out, input[beg:end], tabSize)
+			}
+		}
+
+		if end < len(input) && input[end] == '\r' {
+			end++
+		}
+		if end < len(input) && input[end] == '\n' {
+			end++
+		}
+		out.WriteByte('\n')
+
+		beg = end
 	}

 	// empty input?
@ -635,12 +642,12 @@ func scanLinkRef(p *parser, data []byte, i int) (linkOffset, linkEnd, titleOffse
 		i++
 	}
 	linkOffset = i
-	for i < len(data) && data[i] != ' ' && data[i] != '\t' && data[i] != '\n' && data[i] != '\r' {
-		i++
-	}
 	if i == len(data) {
 		return
 	}
+	for i < len(data) && data[i] != ' ' && data[i] != '\t' && data[i] != '\n' && data[i] != '\r' {
+		i++
+	}
 	linkEnd = i
 	if data[linkOffset] == '<' && data[linkEnd-1] == '>' {
 		linkOffset++
@ -799,7 +806,17 @@ func ispunct(c byte) bool {

 // Test if a character is a whitespace character.
 func isspace(c byte) bool {
-	return c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\f' || c == '\v'
+	return ishorizontalspace(c) || isverticalspace(c)
+}
+
+// Test if a character is a horizontal whitespace character.
+func ishorizontalspace(c byte) bool {
+	return c == ' ' || c == '\t'
+}
+
+// Test if a character is a vertical whitespace character.
+func isverticalspace(c byte) bool {
+	return c == '\n' || c == '\r' || c == '\f' || c == '\v'
 }

 // Test if a character is letter.
--- a/markdown_test.go
+++ b/markdown_test.go
@ -0,0 +1,75 @@
+//
+// Blackfriday Markdown Processor
+// Available at http://github.com/russross/blackfriday
+//
+// Copyright © 2011 Russ Ross <russ@russross.com>.
+// Distributed under the Simplified BSD License.
+// See README.md for details.
+//
+
+//
+// Unit tests for full document parsing and rendering
+//
+
+package blackfriday
+
+import (
+	"testing"
+)
+
+func runMarkdown(input string) string {
+	return string(MarkdownCommon([]byte(input)))
+}
+
+func doTests(t *testing.T, tests []string) {
+	// catch and report panics
+	var candidate string
+	defer func() {
+		if err := recover(); err != nil {
+			t.Errorf("\npanic while processing [%#v]: %s\n", candidate, err)
+		}
+	}()
+
+	for i := 0; i+1 < len(tests); i += 2 {
+		input := tests[i]
+		candidate = input
+		expected := tests[i+1]
+		actual := runMarkdown(candidate)
+		if actual != expected {
+			t.Errorf("\nInput   [%#v]\nExpected[%#v]\nActual  [%#v]",
+				candidate, expected, actual)
+		}
+
+		// now test every substring to stress test bounds checking
+		if !testing.Short() {
+			for start := 0; start < len(input); start++ {
+				for end := start + 1; end <= len(input); end++ {
+					candidate = input[start:end]
+					_ = runMarkdown(candidate)
+				}
+			}
+		}
+	}
+}
+
+func TestDocument(t *testing.T) {
+	var tests = []string{
+		// Empty document.
+		"",
+		"",
+
+		" ",
+		"",
+
+		// This shouldn't panic.
+		// https://github.com/russross/blackfriday/issues/172
+		"[]:<",
+		"<p>[]:&lt;</p>\n",
+
+		// This shouldn't panic.
+		// https://github.com/russross/blackfriday/issues/173
+		"   [",
+		"<p>[</p>\n",
+	}
+	doTests(t, tests)
+}
--- a/ref_test.go
+++ b/ref_test.go
@ -29,7 +29,7 @@ func doTestsReference(t *testing.T, files []string, flag int) {
 	var candidate string
 	defer func() {
 		if err := recover(); err != nil {
-			t.Errorf("\npanic while processing [%#v]\n", candidate)
+			t.Errorf("\npanic while processing [%#v]: %s\n", candidate, err)
 		}
 	}()

--- a/smartypants.go
+++ b/smartypants.go
@ -39,7 +39,7 @@ func isdigit(c byte) bool {
 	return c >= '0' && c <= '9'
 }

-func smartQuoteHelper(out *bytes.Buffer, previousChar byte, nextChar byte, quote byte, isOpen *bool) bool {
+func smartQuoteHelper(out *bytes.Buffer, previousChar byte, nextChar byte, quote byte, isOpen *bool, addNBSP bool) bool {
 	// edge of the buffer is likely to be a tag that we don't get to see,
 	// so we treat it like text sometimes

@ -96,6 +96,12 @@ func smartQuoteHelper(out *bytes.Buffer, previousChar byte, nextChar byte, quote
 		*isOpen = false
 	}

+	// Note that with the limited lookahead, this non-breaking
+	// space will also be appended to single double quotes.
+	if addNBSP && !*isOpen {
+		out.WriteString("&nbsp;")
+	}
+
 	out.WriteByte('&')
 	if *isOpen {
 		out.WriteByte('l')
@ -104,6 +110,11 @@ func smartQuoteHelper(out *bytes.Buffer, previousChar byte, nextChar byte, quote
 	}
 	out.WriteByte(quote)
 	out.WriteString("quo;")
+
+	if addNBSP && *isOpen {
+		out.WriteString("&nbsp;")
+	}
+
 	return true
 }

@ -116,7 +127,7 @@ func smartSingleQuote(out *bytes.Buffer, smrt *smartypantsData, previousChar byt
 			if len(text) >= 3 {
 				nextChar = text[2]
 			}
-			if smartQuoteHelper(out, previousChar, nextChar, 'd', &smrt.inDoubleQuote) {
+			if smartQuoteHelper(out, previousChar, nextChar, 'd', &smrt.inDoubleQuote, false) {
 				return 1
 			}
 		}
@ -141,7 +152,7 @@ func smartSingleQuote(out *bytes.Buffer, smrt *smartypantsData, previousChar byt
 	if len(text) > 1 {
 		nextChar = text[1]
 	}
-	if smartQuoteHelper(out, previousChar, nextChar, 's', &smrt.inSingleQuote) {
+	if smartQuoteHelper(out, previousChar, nextChar, 's', &smrt.inSingleQuote, false) {
 		return 0
 	}

@ -205,13 +216,13 @@ func smartDashLatex(out *bytes.Buffer, smrt *smartypantsData, previousChar byte,
 	return 0
 }

-func smartAmpVariant(out *bytes.Buffer, smrt *smartypantsData, previousChar byte, text []byte, quote byte) int {
+func smartAmpVariant(out *bytes.Buffer, smrt *smartypantsData, previousChar byte, text []byte, quote byte, addNBSP bool) int {
 	if bytes.HasPrefix(text, []byte("&quot;")) {
 		nextChar := byte(0)
 		if len(text) >= 7 {
 			nextChar = text[6]
 		}
-		if smartQuoteHelper(out, previousChar, nextChar, quote, &smrt.inDoubleQuote) {
+		if smartQuoteHelper(out, previousChar, nextChar, quote, &smrt.inDoubleQuote, addNBSP) {
 			return 5
 		}
 	}
@ -224,12 +235,15 @@ func smartAmpVariant(out *bytes.Buffer, smrt *smartypantsData, previousChar byte
 	return 0
 }

-func smartAmp(out *bytes.Buffer, smrt *smartypantsData, previousChar byte, text []byte) int {
-	return smartAmpVariant(out, smrt, previousChar, text, 'd')
-}
+func smartAmp(angledQuotes, addNBSP bool) func(out *bytes.Buffer, smrt *smartypantsData, previousChar byte, text []byte) int {
+	var quote byte = 'd'
+	if angledQuotes {
+		quote = 'a'
+	}

-func smartAmpAngledQuote(out *bytes.Buffer, smrt *smartypantsData, previousChar byte, text []byte) int {
-	return smartAmpVariant(out, smrt, previousChar, text, 'a')
+	return func(out *bytes.Buffer, smrt *smartypantsData, previousChar byte, text []byte) int {
+		return smartAmpVariant(out, smrt, previousChar, text, quote, addNBSP)
+	}
 }

 func smartPeriod(out *bytes.Buffer, smrt *smartypantsData, previousChar byte, text []byte) int {
@ -253,7 +267,7 @@ func smartBacktick(out *bytes.Buffer, smrt *smartypantsData, previousChar byte,
 		if len(text) >= 3 {
 			nextChar = text[2]
 		}
-		if smartQuoteHelper(out, previousChar, nextChar, 'd', &smrt.inDoubleQuote) {
+		if smartQuoteHelper(out, previousChar, nextChar, 'd', &smrt.inDoubleQuote, false) {
 			return 1
 		}
 	}
@ -337,7 +351,7 @@ func smartDoubleQuoteVariant(out *bytes.Buffer, smrt *smartypantsData, previousC
 	if len(text) > 1 {
 		nextChar = text[1]
 	}
-	if !smartQuoteHelper(out, previousChar, nextChar, quote, &smrt.inDoubleQuote) {
+	if !smartQuoteHelper(out, previousChar, nextChar, quote, &smrt.inDoubleQuote, false) {
 		out.WriteString("&quot;")
 	}

@ -367,14 +381,30 @@ type smartCallback func(out *bytes.Buffer, smrt *smartypantsData, previousChar b

 type smartypantsRenderer [256]smartCallback

+var (
+	smartAmpAngled      = smartAmp(true, false)
+	smartAmpAngledNBSP  = smartAmp(true, true)
+	smartAmpRegular     = smartAmp(false, false)
+	smartAmpRegularNBSP = smartAmp(false, true)
+)
+
 func smartypants(flags int) *smartypantsRenderer {
 	r := new(smartypantsRenderer)
+	addNBSP := flags&HTML_SMARTYPANTS_QUOTES_NBSP != 0
 	if flags&HTML_SMARTYPANTS_ANGLED_QUOTES == 0 {
 		r['"'] = smartDoubleQuote
-		r['&'] = smartAmp
+		if !addNBSP {
+			r['&'] = smartAmpRegular
+		} else {
+			r['&'] = smartAmpRegularNBSP
+		}
 	} else {
 		r['"'] = smartAngledDoubleQuote
-		r['&'] = smartAmpAngledQuote
+		if !addNBSP {
+			r['&'] = smartAmpAngled
+		} else {
+			r['&'] = smartAmpAngledNBSP
+		}
 	}
 	r['\''] = smartSingleQuote
 	r['('] = smartParens
--- a/testdata/Links,
+++ b/testdata/Links,
@ -8,4 +8,6 @@

 <p><a href="/url/" title="title has spaces afterward">URL and title</a>.</p>

+<p><a href="/url/">URL with backslashes\</a>.</p>
+
 <p>[Empty]().</p>
--- a/testdata/Links,
+++ b/testdata/Links,
@ -8,5 +8,6 @@ Just a [URL](/url/).

 [URL and title](/url/ "title has spaces afterward"  ).

+[URL with backslashes\\](/url/).

 [Empty]().
Author	SHA1	Message	Date
Dmitri Shuralyov	e96880f42b	README: make a definitive decision on v2 import path This change makes it clear that the v2 import path is github.com/russross/blackfriday/v2, and updates various links accordingly. See https://github.com/russross/blackfriday/issues/587#issuecomment-703393820 for details. This change also converges the README for v1 and v2 to be consistent, as they've started to drift apart. (See PR #675 for the equivalent change to the README on v2 branch.) For #587. GitHub-Pull-Request: #674	2020-10-26 23:46:40 -04:00
Russ Ross	abb995c466	Merge pull request #586 from kolyshkin/gomod go.mod: add go version	2020-03-25 09:18:22 -06:00
Kir Kolyshkin	665c4def65	go.mod: add go version This is a result of running `go mod tidy` using go 1.13.6. In my devel toolchain (vim-go -> gopls -> go 1.13), go apparently wants to add this line to go.mod, but since the modules are read-only, it fails, which leads to gopls failing, too. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>	2020-03-23 19:11:51 -07:00
Russ Ross	41c5fccfd6	Merge pull request #559 from alanhdu/patch-1 Add <details> to recognized HTML5 block tags	2020-02-18 16:49:12 -07:00
Russ Ross	75d2440b2e	Merge pull request #570 from orf/add-no-opener Backport noopener links to v1	2020-02-18 16:47:11 -07:00
Russ Ross	89e6d96148	Merge pull request #580 from MarkAtwood/master Update LICENSE.txt	2020-02-18 16:43:38 -07:00
Mark Atwood	c04ab11830	Update LICENSE.txt License is not changed. Remove line quote characters so that license scanners can recognize the license.	2020-02-03 13:41:32 -08:00
Tom Forbes	41eb1fe8a7	Backport noopener links (from commit `670777b536`)	2019-09-22 15:27:07 +01:00
Alan Du	e2f6cd9f54	Add <summary> tag too	2019-08-20 17:54:39 -04:00
Alan Du	5c6bc5df8a	Add <details> to recognized HTML5 block tags Closes #525	2019-08-20 17:40:27 -04:00
Anatoly Milkov	a925a152c1	README: update URL of LaTeX renderer (#465 ) It has moved from Bitbucket, first to GitHub, then to GitLab. Point to its latest location. Updates #388.	2019-06-16 15:52:46 -04:00
Christian Oliff	f3ccc8fc06	https link to GitHub (#538 )	2019-04-17 22:17:06 +03:00
Ignas Anikevicius	a477dd1646	Fix fenced code in lists (#521 ) This attempts to fix #495 and #485. Note the test cases which were added at the bottom of the list. The first added test case was passing even before the changes, but the second was not.	2019-01-24 10:23:35 +02:00
Iskander (Alex) Sharipov	13768b07fa	readme: use fenced code blocks (#519 ) It's more consistent and it's possible to mark code blocks with "go" hint, so we get syntax highlighting on GitHub.	2019-01-20 20:42:03 +02:00
autopp	8c3eacd7a5	Fix checking of backslash in link text (#510 ) (#512 )	2019-01-20 20:38:36 +02:00
Harald Nordgren	abafa45cd8	Bump Go versions (#499 )	2018-11-15 22:38:43 +02:00
Bjørn Erik Pedersen	05f3235734	Add go.mod (#488 )	2018-09-17 21:59:06 +03:00
Tim Fogarty	f1f45ab762	Fix block parsing for fenced code blocks in lists (#476 ) This is a backport of the initial fix in v2 (#372).	2018-08-29 21:04:01 +03:00
Vytautas Šaltenis	46c73eb196	Add a link to bfchroma, glue library with Chroma (#400 ) Chroma is a general purpose code highlighting library, and bfchroma provides an easy to use glue layer for Blackfriday to interface with Chroma. Fixes #10.	2018-08-04 13:11:49 +03:00
Vytautas Šaltenis	34d6fae961	Don't run tests against ancient versions on Travis (#438 ) Don't run tests against ancient versions on Travis.	2018-08-04 13:08:49 +03:00
Nathan Glenn	11635eb403	Accept info strings in code fences (#448 ) * Accept info strings in code fences According to the common mark standard, code fence info strings can be anything, not just single words. Update the tests and parser accordingly. The formatter already expected an info string with a language and HTML classes, so this does not need to change. Update the LaTeX formatter to take the first word of the info string as the language. Fixes #410 (in v1). * Don't output whole info string as code classes This follows the common mark specification. * run go fmt	2018-04-28 13:25:19 +03:00
Yoshiya Hinosawa	16ac584625	docs(README.md): fix typo (#451 )	2018-04-26 21:50:03 +03:00
Vytautas Šaltenis	55d61fa8aa	Merge pull request #433 from IainHaslam/master Fix HTML5 by removing rel="footnote" which is invalid.	2018-02-18 11:36:01 +02:00
Iain Haslam	338ca359c1	Fix HTML5 by removing rel="footnote" which is invalid. Resolves #336.	2018-02-16 22:21:04 +00:00
Vytautas Šaltenis	8db9055e38	Merge pull request #432 from vassudanagunta/patch-1 Fix broken link to v2 in README	2018-02-12 10:28:20 +02:00
Vas Sudanagunta	e32caebbc7	Fix broken link to v2 in README	2018-02-07 15:16:47 -05:00
Vytautas Šaltenis	6d1ef893fc	Merge pull request #354 from russross/readme-for-v2 Document V2 in master README	2017-10-11 21:22:19 +03:00
Vytautas Šaltenis	8249792ba8	Update on import paths and dep transitive import FAQ Update usage and import paths with and without package management. Document the dep transitive dependency issue and how to get around it. Couple more minor edits.	2017-10-07 12:41:58 +03:00
Vytautas Šaltenis	8ad7e40db0	Update README with the latest changes from v2	2017-10-07 11:14:28 +03:00
Vytautas Šaltenis	a03b7ee643	Add v1 import path, fix Bluemonday code snippet	2017-10-07 11:14:28 +03:00
Vytautas Šaltenis	52dd06192f	Badges back to heading, reference v1 docs explicitly	2017-10-07 11:14:28 +03:00
Vytautas Šaltenis	ad4c953876	Put row of badges above the top level heading	2017-10-07 11:14:28 +03:00
Vytautas Šaltenis	74be510331	Document V2 in master README	2017-10-07 11:14:28 +03:00
Vytautas Šaltenis	4048872b16	Merge pull request #379 from bep/guillemets Add Smartypants support for French Guillemets	2017-07-28 20:53:26 +03:00
Bjørn Erik Pedersen	4ca8c28b21	Add Smartypants support for French Guillemets This commits adds flag `HTML_SMARTYPANTS_QUOTES_NBSP` which, when combined with `HTML_USE_SMARTYPANTS` will insert non-breaking spaces between the double quotes and the contained text. This is mostly relevant for use in French with `HTML_SMARTYPANTS_ANGLED_QUOTES`. It should not hurt existing code path in the performance department: ``` name old time/op new time/op delta SmartDoubleQuotes-4 2.58µs ± 1% 2.58µs ± 1% ~ (p=1.000 n=5+5) name old alloc/op new alloc/op delta SmartDoubleQuotes-4 5.27kB ± 0% 5.27kB ± 0% ~ (all samples are equal) name old allocs/op new allocs/op delta SmartDoubleQuotes-4 13.0 ± 0% 13.0 ± 0% ~ (all samples are equal) ``` Fixes #378	2017-07-27 22:08:20 +02:00
Vytautas Šaltenis	067529f716	Merge pull request #366 from choueric/master fix duplicate and recursive footnotes. (#241)	2017-06-10 20:02:32 +03:00
choueric	5b2fb1b893	use map[string]struct{} as a set. Use map[string]struct{} instead of map[string]bool to implement a set and reduce memory space.	2017-06-10 22:24:27 +08:00
choueric	8c89af6200	add 'notesRecord' to check footnote existence fast It is necessary to use vector for 'notes' instead of map to keep footnotes ordered. But checking for presence in a vector is not efficient. So add a map variable 'notesRecord' to tackle this problem.	2017-06-09 10:20:58 +08:00
choueric	a20399916c	fix duplicate and recursive footnotes. (#241 ) Fix the infinite loop when there is a self-refer footnote by checking if the reference is already added into parser.notes. It also fixes of creating duplicate footnotes through this modification. Add coresponding testcase.	2017-06-08 14:43:56 +08:00
Dmitri Shuralyov	0ba0f2b6ed	Document SanitizedAnchorName algorithm, copy implementation. (#352 ) The goal of this change is to reduce number of non-standard library packages (repositories) that blackfriday imports from 1 to 0, and in turn, reduce the cost of importing blackfriday into other projects. Do so by documenting the algorithm of SanitizedAnchorName, and include a copy of the small function inside blackfriday itself. The same functionality continues to be available in the original location, github.com/shurcooL/sanitized_anchor_name.Create. It can be used by existing users and those that look for a small package, and don't need all of blackfriday functionality. Existing users of blackfriday can use the new SanitizedAnchorName function directly and avoid an extra package import. Resolves #350.	2017-05-09 02:07:14 -04:00
Vytautas Šaltenis	b253417e1c	Merge pull request #334 from choueric/master add an extension to handle Chinese (or CJK) newlines.	2017-04-13 20:36:32 +03:00
choueric	f0bb45f44c	modify testcase of joinLine. 1. delete testdata 'zhJoinLines.txt'. 2. move the testcase from markdown_test.go into block_test.go.	2017-03-17 17:21:30 +08:00
choueric	8098dab4eb	add testcase for joinLines extension	2017-02-22 09:18:21 +08:00
choueric	3ffe8c7f6b	add extension to join lines	2017-02-22 09:15:46 +08:00
Kévin Dunglas	5f33e7b787	Document using fenced code blocks with bluemonday (#309 ) Document using fenced code blocks with bluemonday	2016-10-03 19:27:22 +03:00
Dmitri Shuralyov	35eb537633	Set LIST_ITEM_END_OF_LIST when list is at end of document. (#305 ) The LIST_ITEM_END_OF_LIST flag is an internal flag passed to renderers when rendering list items. It's normally set for the last item in the list. This change fixes the issue where that flag wasn't set in situations where the Markdown document ends with the list being the last block level item in it. The cases above detect and set LIST_ITEM_END_OF_LIST flag when the list ends because another thing begins, but they miss it when the end of document is reached. No tests here because this subtle internal behavior is hard to test and would require quite a bit of testing/mock infrastructure. Helps shurcooL/markdownfmt#30.	2016-09-08 23:28:54 -07:00
Dmitri Shuralyov	93622da34e	Make newline mandatory for opening fence line. (#281 ) This was an unintended typo/mistake in #280. This is stricter, and it's fine. The opening fence line will always need to have a newline. Add another test for isFenceLine.	2016-07-16 11:34:03 -04:00
Vytautas Šaltenis	4e6f303e8d	Merge pull request #280 from russross/fix-279-and-refactor Fix fenced code block rendering with content resembling references.	2016-07-15 22:37:58 +03:00
Dmitri Shuralyov	a5812bb8f2	Improve fenced code block detection for first pass. In first pass, there may not be a trailing newline after a fenced code block yet. Make newline optional in isFenceLine when calling fencedCodeBlock to detect the fenced code block it anyway. This is more complex, but it avoids creating temporary buffers or modifying input in order to maintain performance (see #148). Document and rename fencedCode to fencedCodeBlock. Add regression tests. Fixes #279.	2016-07-15 15:07:43 -04:00
Dmitri Shuralyov	0049676599	Improve fence line detection. Rename isFenceCode to isFenceLine, document it, add tests. Add support for making newline optional, this will be needed in future commits.	2016-07-15 14:59:57 -04:00
Dmitri Shuralyov	96537c6eaa	Create a place for document-level unit tests. These will be helpful for catching regressions or changes in behavior to edge cases such as empty input, or specifically crafted inputs that may cause panics, etc. Move test for issue #172 there since it's a document-level test, not an inline one. Add test for issue #173. Make some things more consistent. Don't use a named receiver in methods that don't use it. This makes the code more readable since one can more quickly tell the inputs to the method.	2016-07-15 14:59:57 -04:00
Vytautas Šaltenis	1d6b8e9301	Merge pull request #266 from russross/avoid-unneeded-break Avoid unneeded break statement.	2016-05-31 14:12:24 +03:00
Dmitri Shuralyov	b88a9bd458	Avoid unneeded break statement. This is purely a style change with no behavior difference. In Go (unlike most other languages), case statements in a switch don't need an explicit break statement, it happens by default. Adding it explicitly is possible, but has no effect. In this case, having the break statement hurts readability because it's hard to tell if it's a mistake, and the break was intended to break out of the outer for loop, rather than do nothing for the switch statement. So, remove it, to make it more clear that there is no bug here.	2016-05-24 22:04:36 -07:00
Vytautas Šaltenis	2004188462	Merge pull request #264 from moorereason/iss263 Fix definition lists that contain other lists	2016-05-10 22:09:22 +03:00
Cameron Moore	acc07e6144	Fix definition lists that contain other lists Fixes #263	2016-05-06 13:27:55 -05:00
Vytautas Šaltenis	43529be397	Merge pull request #261 from moorereason/iss235 Fix adjacent list merging	2016-05-02 20:28:48 +03:00
Cameron Moore	0529888f55	Fix adjacent list merging Prevent adjacent lists of differing types from being merged into a single list. No options are provided to enable the previous behavior. Fixes #235	2016-05-02 09:02:35 -05:00
Dmitri Shuralyov	fc997ac2ba	Travis: Run gofmt -s, go vet, go test -race, add Go 1.6. Also make tip a fast-finish allowed failure. That way, if CI fails on tip due to a temporary issue with tip, it will not break build status. However, it's still possible to see tip build status by looking at CI details page. Do not run go vet with Go 1.4 or older since it's not included in the standard library, and it's no longer available in external standard library. Add godoc badge to README.md.	2016-04-28 22:47:04 -07:00
Vytautas Šaltenis	151efb040f	Merge pull request #250 from tomkwok/master Fix regession: index out of range panic in reference link (#172, #173)	2016-04-21 11:09:30 +03:00
Tom Kwok	69f51afe42	Add test for issue 172	2016-04-03 21:08:32 +08:00
Tom Kwok	232d06cf99	Fix regression: index out of range panic in reference link (#172 , #173 )	2016-04-02 22:13:43 +08:00
Vytautas Šaltenis	b43df972fb	Merge pull request #247 from shawnps/patch-1 fix typo	2016-03-16 09:29:03 +02:00
Shawn Smith	1d94959fea	fix typo	2016-03-16 15:21:50 +09:00
Vytautas Šaltenis	006144af03	Merge pull request #234 from shawnps/gofmt gofmt -s	2016-01-24 13:12:56 +02:00
shawnps	2ee51ae2a0	gofmt -s	2016-01-22 16:04:49 -08:00
Vytautas Šaltenis	c8875c0ed4	Merge pull request #229 from russross/issue-228 Fix bug with gathering list item lines	2015-12-30 17:42:28 +02:00
Vytautas Šaltenis	fc6236fc55	Fix bug with gathering list item lines Instead of swallowing an empty line and then reintroducing it back again in certain cases, collect the list item body in an unaltered form and let the recursive parsing call sort things out. Fixes issue #228.	2015-12-26 20:18:22 +02:00
Vytautas Šaltenis	b8031576aa	Merge pull request #225 from russross/issue-165 Implement support for CDATA section	2015-12-25 13:14:44 +02:00
Vytautas Šaltenis	ee63ffd3e2	Fix initialisms in function names	2015-12-25 13:04:56 +02:00
Vytautas Šaltenis	f5ac2ddf7a	Add a few more CDATA tests	2015-12-16 20:53:40 +02:00
Vytautas Šaltenis	594d923645	Convert constant variables into consts	2015-12-16 20:51:44 +02:00
Vytautas Šaltenis	05a79218ed	Implement support for CDATA section Fixes #165.	2015-12-14 20:57:40 +02:00
Vytautas Šaltenis	d18b67ae0a	Add more tests for emphasised text Including EXTENSION_NO_INTRA_EMPHASIS. These nail down a fix for #178 that was a side effect of `cc3cc10`.	2015-12-12 19:18:59 +02:00
Vytautas Šaltenis	3c4a488ada	Merge pull request #224 from russross/issue-180 Fix for #180	2015-12-11 20:25:12 +02:00
Vytautas Šaltenis	fc740701cf	Cosmetics: fix comment placement and import order	2015-12-10 21:03:21 +02:00
Vytautas Šaltenis	bcd6dd8711	Don't require the input to end with a newline Technically, not ending a line with a newline is wrong, but this blunder is so pervasive that we have no other choice but to live with it. Fixes #180.	2015-12-10 20:52:47 +02:00
Vytautas Šaltenis	e3cbadc5c9	Bump version	2015-12-06 19:26:14 +02:00
Vytautas Šaltenis	ce3ffa70a0	Merge pull request #222 from icco/parens Allow parentheses inside of links	2015-12-06 18:44:17 +02:00
Nat Welch	1bb1fa9e95	remove check that brace is > 0	2015-12-05 02:00:36 +00:00
Nat Welch	c1d4a77100	Fix comment formatting and spelling	2015-12-05 02:00:23 +00:00
Nat Welch	45f5800b75	Get tests to pass	2015-11-29 00:08:48 +00:00
Nat Welch	1622f8f312	Apply @miekg's patch From `99ce7134f6`.patch	2015-11-26 22:11:25 +00:00
Vytautas Šaltenis	300106c228	Merge pull request #219 from FooSoft/master Adding a link to Md2Vim.	2015-11-17 09:23:12 +02:00
Alex Yatskov	7d22880556	Adding a link to Md2Vim.	2015-11-17 12:58:23 +09:00