feat(yard): add initial support of YARD tags #1416

okuramasafumi · 2025-08-27T08:44:08Z

At this time, @yield, @private and @api private are supported.
Ref: #1344

okuramasafumi · 2025-08-27T08:45:28Z

lib/rdoc/yard.rb

+  # Extract parameter names from YARD type specification
+  # e.g., "[String, Integer]" -> "value1, value2"
+  # e.g., "[item, index]" -> "item, index"
+  def extract_param_names(params_string)


This part is kind of too much, but YARD's @yield might cause confusion to use type name instead of param name. Here it replaces type name with temporary param name,

At this time, `@yield`, `@private` and `@api private` are supported.

tompng · 2025-08-29T14:37:28Z

lib/rdoc/yard.rb

+  ##
+  # Extract parameter names from YARD type specification
+  # e.g., "[String, Integer]" -> "value1, value2"
+  # e.g., "[item, index]" -> "item, index"


Example and implementation does not match.
"String, Integer" -> "arg1, arg2"
"item, index" -> "item, index"

tompng · 2025-08-29T14:45:18Z

lib/rdoc/yard.rb

+    return if code_object.block_params && !code_object.block_params.empty?
+
+    # Match @yield [params] description
+    if text =~ /^\s*#?\s*@yield\s*(?:\[([^\]]*)\])?\s*(.*?)$/


YARD tag have a common format # @tagname free_format_argument_part # @!tagname free_format_argument_part
First scan tags in a generic way, and pass it to each tag-process part is better.
This way, parsing multiline tags would be easier.

tompng · 2025-08-29T14:50:55Z

lib/rdoc/code_object/method_attr.rb

+  def comment=(comment)
+    # Process YARD tags while we still have the RDoc::Comment object
+    if comment.is_a?(RDoc::Comment)
+      RDoc::YARD.process(comment, self)


In the comment below, @foo is not a YARD tag.

# @foo &&= bar # ^^^^^^^^^^^^ def visit_instance_variable_and_write_node(node) end

Ruby's comment frequently contains code example that uses instance variable. It's the same format as YARD tags.

So we need to make YARD feature opt-in, or only accept specific yard tags. (opt-in is better)

# This is just an idea. Specify that this file uses YARD format. # :markup: yard class A # @yard-tag def foo; end end

# found 37 files in github code search # @api = ...

tompng · 2025-08-29T14:52:28Z

lib/rdoc/parser/prism_ruby.rb

      meth,
      line_no: start_line,
-      visibility: visibility,
+      visibility: visibility,  # Use YARD visibility if set, otherwise use Ruby visibility


This comment should be where YARD actually overrides visibility, near visibility = meth.visibility line.

tompng · 2025-08-29T15:45:26Z

lib/rdoc/yard.rb

+
+  ##
+  # Process @api tags (specifically @api private)
+  def process_api_tags


I think @api private sets documenting level visibility which RDoc doesn't have. It's not for making a method private_method.
One idea is to let it act as nodoc for now.

tompng · 2025-08-29T16:14:43Z

lib/rdoc/yard.rb

+    return comment unless comment && code_object
+
+    # Only process RDoc::Comment objects to avoid modifying plain strings
+    # that aren't actually documentation comments
+    return comment unless comment.is_a?(RDoc::Comment)


These guards are not needed. Method signature will be simple.

(String | RDoc::Comment | nil, RDoc::CodeObject | nil) -> void # ↓ (RDoc::Comment, RDoc::CodeObject) -> void

RDoc::Parser::PrismRuby don't set String as comment, and it will be the default in the future.

tompng · 2025-08-29T16:48:09Z

lib/rdoc/code_object/constant.rb


+  ##
+  # Process comment as YARD comment
+  def comment=(comment)


Handling YARD in comment= seems too aggressive. It can be triggered from many different situation and places.

def add_alias # Maybe this is ok, but I can't confidently say that this is safe method.comment = an_alias.comment end

How about explicitly processing it in RDoc::Parser::PrismRuby?

def add_comment ... comment.normalize override_visibility, some_other_information = handle_yard(meth, comment) if override_visibility do_something end meth.comment = comment ... end

RDoc::Parser::Ruby is too complicated (and buggy) to maintain, so the system design of YARD-support shouldn't be affected by RDoc::Parser::Ruby implementation detail. I think supporting YARD only in RDoc::Parser::PrismRuby is enough.

kou · 2025-08-29T22:00:07Z

lib/rdoc/parser/prism_ruby.rb

+      # Save visibility before comment assignment (where YARD processing happens)
+      visibility_before = meth.visibility
      meth.comment = comment
+      # Check if YARD tags changed the visibility
+      if meth.visibility != visibility_before
+        visibility = meth.visibility
+      end


It seems that this shows that the proposed approach isn't integrated with the current implementation clearly. Can we use another more integrated approach?

kou · 2025-08-29T22:03:40Z

lib/rdoc/yard.rb

+# - @private - Marks method/class as private
+# - @api private - Alternative private marking
+
+class RDoc::YARD


Can we improve this name?

I think that "YARD" isn't suitable here because we don't support full YARD features. We only focus on YARD tags here.

kou · 2025-08-29T22:04:05Z

lib/rdoc/yard.rb

+    new(comment, code_object).process
+  end
+
+  attr_reader :comment, :code_object, :text


Do we need them?

kou · 2025-08-29T22:04:32Z

lib/rdoc/yard.rb

+  def self.process(comment, code_object)
+    new(comment, code_object).process
+  end


Do we need this shortcut?

kou · 2025-08-29T22:08:50Z

lib/rdoc/yard.rb

+    return comment unless comment.is_a?(RDoc::Comment)
+
+    # Skip processing if comment is in TomDoc format
+    return comment if comment.format == 'tomdoc'


Can we use comment.tomdoc?

BTW, is this a responsibility of this class? It seems that this is a responsibility of callers.

kou · 2025-08-29T22:36:17Z

lib/rdoc/yard.rb

+    return if code_object.block_params && !code_object.block_params.empty?
+
+    # Match @yield [params] description
+    if text =~ /^\s*#?\s*@yield\s*(?:\[([^\]]*)\])?\s*(.*?)$/


@yield can use multiple lines: https://github.com/lsegal/yard/blob/5b93b3a4f15a52f3165edef112e718149d294c38/lib/yard/parser/source_parser.rb#L223-L224

kou · 2025-08-29T22:37:46Z

lib/rdoc/yard.rb

+  def process_private_tags
+    return unless code_object.respond_to?(:visibility=)
+
+    if text =~ /^\s*#?\s*@private\s*$/


Why is ? needed after #?

kou · 2025-08-29T22:38:57Z

lib/rdoc/yard.rb

+  # e.g., "[String, Integer]" -> "value1, value2"
+  # e.g., "[item, index]" -> "item, index"
+  def extract_param_names(params_string)
+    return '' if params_string.nil? || params_string.empty?


Do we need these checks?

I feel that it's responsibility of callers.

kou · 2025-08-29T22:40:44Z

lib/rdoc/yard.rb

+    return '' if params_string.nil? || params_string.empty?
+
+    # If params look like variable names (lowercase start), use them
+    if params_string =~ /^[a-z_]/


We should use \A not ^ for the start of string. ^ is for the start of line.
See also: https://www.clear-code.com/blog/2015/3/25.html (Japanese)

kou · 2025-08-29T22:41:44Z

lib/rdoc/yard.rb

+
+    # If params look like variable names (lowercase start), use them
+    if params_string =~ /^[a-z_]/
+      params_string.split(',').map(&:strip).join(', ')


Suggested change

params_string.split(',').map(&:strip).join(', ')

params_string.split(/\s*,\s*/).join(', ')

okuramasafumi commented Aug 27, 2025

View reviewed changes

okuramasafumi requested a deployment to fork-preview-protection August 27, 2025 08:45 — with GitHub Actions Waiting

okuramasafumi requested a deployment to fork-preview-protection August 28, 2025 03:09 — with GitHub Actions Waiting

okuramasafumi added 2 commits August 28, 2025 14:53

feat(yard): add initial support of YARD tags

450df2f

At this time, `@yield`, `@private` and `@api private` are supported.

fix(yard): YARD parsing now works with Prism parser

8bc0794

okuramasafumi force-pushed the add-yard-yield-private-tag branch from 1ca2295 to 8bc0794 Compare August 28, 2025 05:53

okuramasafumi requested a deployment to fork-preview-protection August 28, 2025 05:53 — with GitHub Actions Waiting

tompng reviewed Aug 29, 2025

View reviewed changes

kou reviewed Aug 29, 2025

View reviewed changes

	params_string.split(',').map(&:strip).join(', ')
	params_string.split(/\s,\s/).join(', ')

feat(yard): add initial support of YARD tags #1416

Are you sure you want to change the base?

feat(yard): add initial support of YARD tags #1416

Conversation

okuramasafumi commented Aug 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!