Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding remove_obsidian_comments postprocessor to remove obsidian format comments #268

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,30 @@ With these hooks in place, links to both notes as well as file attachments shoul
>
> Note: If you're using a theme which comes with it's own render hooks, you might need to do a little extra work, or customize the snippets above, to avoid conflicts with the hooks from your theme.

## Removing Obsidian comments

To remove [Obsidian formatted comments](https://help.obsidian.md/Editing+and+formatting/Basic+formatting+syntax#Comments) you can set the `--remove-obsidian-comments` flag. This will remove inline as well as block comments but leave any comments found in codeblocks.

````md
This is not a comment.

%% This is an inline comment %% This line could have more text.

%%
This would be a
block comment and
all of this would be removed
%%
````

With the `--remove-obsidian-comments` flag. This file would output as

````md
This is not a comment

This line could have more text.
````


# Library usage

Expand Down
27 changes: 27 additions & 0 deletions docs/usage-advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,30 @@ With these hooks in place, links to both notes as well as file attachments shoul
[hugo-relative-linking]: https://notes.nick.groenen.me/notes/relative-linking-in-hugo/
[hugo]: https://gohugo.io
[markdown render hooks]: https://gohugo.io/getting-started/configuration-markup#markdown-render-hooks

## Removing Obsidian comments

By default, comments are kept unchanged in the output.

To remove [Obsidian formatted comments](https://help.obsidian.md/Editing+and+formatting/Basic+formatting+syntax#Comments) you can set the `--comments=remove` flag. This will remove inline as well as block comments but leave any comments found in codeblocks.

```md
This is not a comment.

%% This is an inline comment %% This line could have more text.

%%
This would be a
block comment and
all of this would be removed
%%
```
With the `--comments=remove` flag. This file would output as

```md
This is not a comment

This line could have more text.
```

> Note: Comments within codeblocks are left untouched
28 changes: 27 additions & 1 deletion src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,12 @@ use std::path::PathBuf;

use eyre::{eyre, Result};
use gumdrop::Options;
use obsidian_export::postprocessors::{filter_by_tags, softbreaks_to_hardbreaks};
use obsidian_export::postprocessors::{
filter_by_tags,
remove_obsidian_comments,
softbreaks_to_hardbreaks,
CommentStrategy,
};
use obsidian_export::{ExportError, Exporter, FrontmatterStrategy, WalkOptions};

const VERSION: &str = env!("CARGO_PKG_VERSION");
Expand Down Expand Up @@ -70,6 +75,15 @@ struct Opts {
default = "false"
)]
hard_linebreaks: bool,

#[options(
no_short,
help = "Comment strategy (one of: keep-unchanged or remove)",
long = "comments",
parse(try_from_str = "comment_strategy_from_str"),
default = "keep-unchanged"
)]
comment_strategy: CommentStrategy,
}

fn frontmatter_strategy_from_str(input: &str) -> Result<FrontmatterStrategy> {
Expand All @@ -81,6 +95,14 @@ fn frontmatter_strategy_from_str(input: &str) -> Result<FrontmatterStrategy> {
}
}

fn comment_strategy_from_str(input: &str) -> Result<CommentStrategy> {
match input {
"keep-unchanged" => Ok(CommentStrategy::KeepUnchanged),
"remove" => Ok(CommentStrategy::Remove),
_ => Err(eyre!("must be one of: keep-unchanged or remove")),
}
}

fn main() {
// Due to the use of free arguments in Opts, we must bypass Gumdrop to determine whether the
// version flag was specified. Without this, "missing required free argument" would get printed
Expand Down Expand Up @@ -111,6 +133,10 @@ fn main() {
exporter.add_postprocessor(&softbreaks_to_hardbreaks);
}

if matches!(args.comment_strategy, CommentStrategy::Remove) {
exporter.add_postprocessor(&remove_obsidian_comments);
}

let tags_postprocessor = filter_by_tags(args.skip_tags, args.only_tags);
exporter.add_postprocessor(&tags_postprocessor);

Expand Down
93 changes: 92 additions & 1 deletion src/postprocessors.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
//! A collection of officially maintained [postprocessors][crate::Postprocessor].

use pulldown_cmark::Event;
use std::cell::LazyCell;

use pulldown_cmark::{CowStr, Event, Tag};
use regex::Regex;
use serde_yaml::Value;

use super::{Context, MarkdownEvents, PostprocessorResult};
Expand Down Expand Up @@ -52,6 +55,94 @@ fn filter_by_tags_(
}
}

//Available strategies for what to do with comments
#[derive(Debug)]
#[non_exhaustive]
pub enum CommentStrategy {
// Leave comments alone and export them as normal
KeepUnchanged,
// Remove any comments from the output
Remove,
}

#[allow(clippy::arithmetic_side_effects)]
/// This postprocessor removes all Obsidian comments from a file excluding codeblocks. Enabling this
/// prohibits comments from being exported but leaves them untouched in the original files
pub fn remove_obsidian_comments(
_context: &mut Context,
events: &mut MarkdownEvents<'_>,
) -> PostprocessorResult {
let mut output = Vec::with_capacity(events.len());
let mut inside_comment = false;
let mut inside_codeblock = false;
let re = LazyCell::new(|| Regex::new(r"%%.*?%%").unwrap());

for event in &mut *events {
output.push(event.to_owned());

match event {
Event::Text(ref text) => {
if !text.contains("%%") {
if inside_comment {
output.pop(); //Inside block comment so remove
}
continue;
}

if inside_codeblock {
continue; //Skip anything inside codeblocks
}

output.pop();

if inside_comment {
inside_comment = false;
continue;
}

if !text.eq(&CowStr::from("%%")) {
let result = re.replace_all(text, "").to_string();
output.push(Event::Text(CowStr::from(result)));
continue;
}

inside_comment = true;
}
Event::Start(Tag::CodeBlock(_)) => {
if inside_comment {
output.pop();
} else {
inside_codeblock = true;
}
}
Event::End(Tag::CodeBlock(_)) => {
if inside_comment {
output.pop();
} else {
inside_codeblock = false;
}
}
Event::End(Tag::Paragraph) => {
if output.len() >= 2
&& output.get(output.len() - 2) == Option::from(&Event::Start(Tag::Paragraph))
{
// If the comment was the only item on the line remove the start and end
// paragraph events to remove the \n in the output file.
output.pop();
output.pop();
}
}
_ => {
if inside_comment {
output.pop();
}
}
}
}
*events = output;
PostprocessorResult::Continue
}

#[test]
fn test_filter_tags() {
let tags = vec![
Expand Down
23 changes: 22 additions & 1 deletion tests/postprocessors_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@ use std::fs::{read_to_string, remove_file};
use std::path::PathBuf;
use std::sync::Mutex;

use obsidian_export::postprocessors::{filter_by_tags, softbreaks_to_hardbreaks};
use obsidian_export::postprocessors::{
filter_by_tags,
remove_obsidian_comments,
softbreaks_to_hardbreaks,
};
use obsidian_export::{Context, Exporter, MarkdownEvents, PostprocessorResult};
use pretty_assertions::assert_eq;
use pulldown_cmark::{CowStr, Event};
Expand Down Expand Up @@ -295,3 +299,20 @@ fn test_filter_by_tags() {
);
}
}

#[test]
fn test_remove_obsidian_comments() {
let tmp_dir = TempDir::new().expect("failed to make tempdir");
let mut exporter = Exporter::new(
PathBuf::from("tests/testdata/input/remove-comments/"),
tmp_dir.path().to_path_buf(),
);
exporter.add_postprocessor(&remove_obsidian_comments);
exporter.run().unwrap();

let expected =
read_to_string("tests/testdata/expected/remove-comments/test_comments.md").unwrap();
let actual = read_to_string(tmp_dir.path().join(PathBuf::from("test_comments.md"))).unwrap();

assert_eq!(expected, actual);
}
6 changes: 6 additions & 0 deletions tests/testdata/expected/remove-comments/test_comments.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Not a comment


````md
%% Comment in code block should be kept %%
````
27 changes: 27 additions & 0 deletions tests/testdata/input/remove-comments/test_comments.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
Not a comment
%% Comment should be removed %%
```md
%% Comment in code block should be kept %%
```

%%
This is
a block comment
that should be removed
%%

%%

None of the below should be visible in output:

## Header

Code block:

```
foo
```

---

%%
Loading