Merge tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-tracegrafted

Pull tracing fixes from Steven Rostedt:

 - Don't destroy user event fields when removal fails

   User event fields are destroyed before the event is removed from
   visibility. But that can fail leaving the still visible event with no
   fields. Move the destroying of the fields to after the event is
   successfully removed from visibility.

 - Initialize function graph state is fork before calling
   copy_exec_state()

   For non-CLONE_VM forks, copy_exec_state() allocates a new
   task_exec_state. If that allocation fails, ftrace_graph_exit_task()
   will free the tasks ret_stack pointer. Since that pointer is still
   using the parent's ret_stack, it mistakenly frees the parent's
   pointer too.

   Call ftrace_graph_init() on the task first which will NULL out the
   new tasks's ret_stack and if the copy fails, it will not free
   anything.

 - Remove FGRAPH_MAX_INDEX

   The macro FGRAPH_MAX_INDEX was added but never used. Remove it.

 - Save ent_size in function graph printing of nested functions

   The function graph tracer needs to look at the next event to see if
   the next event is the return of the current function entry. If it is,
   it prints a single line:

	ktime_get();

   Otherwise it prints it like a nested function:

	tick_nohz_irq_exit() {
	    ktime_get();
	    kcpustat_irq_exit();
	}

   In order to look at the next event, it must save the current event so
   that it has the information to print from it. It saves the event in
   the iterator descriptor called "ent". What it doesn't save is the
   ent_size of the event which is now used to know if the function graph
   arguments are to be printed. The peek doesn't save the size so the
   size used happens to be that of the size of the last event that was
   seen.

   Save the entry event size in the iterator descriptor so that the
   correct size is used.

 - Fix several errors with freeing data in the histogram code

   The histogram code had a lot of leaked or or incorrect accounting
   when failures happen. Correct them.

 - Fix histogram regression of .percent and .graph modifiers

   Up until 6.3 histogram values could have "percent" or "graph"
   modifiers that changed how they were printed. But a change that added
   restricting histograms values from being strings, stack traces and
   other modifiers inadvertently prevented them from using the percent
   and graph modifiers, which were legal use cases for values.

   Put back the percent and graph modifiers.

 - Fix various typos in the comments

 - Set the trace_clock before initializing a histogram with clock
   argument

   The histogram API allows the user to specific which trace clock to
   use via a "clock=" string. The histogram is set up first before the
   clock is checked. If the passed in clock is not valid, it exits
   without fully fixing up the histogram leaving it on the list and a
   use-after-free can trigger.

   Update the clock argument first and if it fails then exit gracefully
   before the histogram trigger is placed on any lists.

 - Restore :mod: trailer after parsing in ftrace_set_clr_event

   The function ftrace_set_clr_event() modifies the parse string and
   needs to put it back to what was passed in. It searches for ":mod:"
   via a strsep() but fails to put back the first ':' in the string.

   Add back the ':' in the passed in string.

 - Take trace_array reference when opening a tracer options file

   The options files are dynamically created and some tracers add their
   own options. When a tracer adds their own list of options, the
   trace_array holding them has an array to hold the list of options for
   each tracer. This array increases in size via a krealloc(), and the
   new entry gets a newly allocated array to hold the options of the new
   tracer being added.

   The element in each entry of the tracer's option array holds a
   pointer back to the trace_array, a pointer to the tracer it is
   associated to, a pointer to the flags of the option.

   The issue is that these arrays are freed when the trace_array is
   freed when its instance it represents is removed from the instances
   directory. There's a race that an open of one of these options files
   can happen when the instance is being removed.

   Add a new helper function to be called by the open function of the
   options file to iterate all existing trace_arrays under a lock and
   find the one that has the given option element in one of it's tracer
   arrays. If found, then update the associated trace_array's reference
   counter to keep it from being freed. If not found, have the open call
   return -ENODEV.

 - Disable interrupts when acquiring the lock in rb_wake_up_waiters()

   The function rb_wake_up_waiters() assumes it will be called in
   interrupt context and does not disable irqs when taking
   cpu_buffer->reader_lock, which can be called in hard interrupt
   context. The issue is in PREEMPT_RT, this function is called in
   thread context leaving this lock open to a deadlock.

   Take the lock with interrupts disabled.

 - Use rcu_assign_pointer() for tmp_ops filter hash

   The tmp_ops used in update_ftrace_direct_mod() assigns its
   filter_hash field directly, but that field is annotated as __rcu and
   sparse complains. Assign it with rcu_assign_pointer()

 - Fix use-after-free in enable_trigger_private_data_free()

   The trace_event_call is accessed through the event_trigger_data's
   trace_event_file pointer to put the trace_event_call on freeing. The
   issue is that the trace_event_file data may have been freed already
   causing a use-after-free. Add a field to the event_trigger_data that
   points directly to the trace_event_call so that it can decrement its
   reference directly without needing to go through the
   trace_event_file.

 - Fix accounting of buffer data remote headers

   trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount
   the number of pages is needed for the asked for size as it doesn't
   take into account the meta data on each page. Add a helper function
   to do the calculation properly and use that in these functions.

 - Catch nr_page_va overflow in ring_buffer_desc sizing

   The number of pages per remote ring buffer is capped by
   ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
   overflow that field would silently allocate a descriptor smaller than
   what was asked for.

 - Do not resize the subbuf order if any per_cpu buffer is disabled

   The mmapping of ring buffers disables resizing the subbuffers, but it
   is done per-cpu whereas the subbuf size change is done for all the
   per_cpu buffers under the buffer->mutex. It could change the size of
   some while the mapping is happening on others. Have the resize of the
   subbuf order check all the per_cpu buffers under the lock to see if
   any of them is disabled before starting and causing an inconsistency
   between buffers that are being mapped.

* tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (25 commits)
  ring-buffer: Check resize_disabled before publishing the new subbuf order
  tracing/remotes: Catch nr_page_va overflow in ring_buffer_desc sizing
  tracing/remotes: Account for ring buffer page header in size calculation
  tracing: Don't dereference trace_event_file in deferred trigger free
  ftrace: Use rcu_assign_pointer() for tmp_ops filter hash
  ring-buffer: Acquire the lock with irqsave in rb_wake_up_waiters()
  tracing: Take trace_array reference when opening a tracer options file
  tracing: Fix ring_buffer_read_page_size() kernel-doc
  tracing: Restore :mod: trailer after parsing in ftrace_set_clr_event()
  tracing: Fix memory corruption from a "STACKTRACE" histogram key
  tracing: Fix memory corruption from the histogram stacktrace modifier
  tracing: Undo the registration when enabling the histogram trigger fails
  tracing: Take the reference before publishing the named histogram trigger
  tracing: Set the trace clock before registering the histogram trigger
  tracing: Fix typo "preceeded" in comment
  tracing: Fix typo "availabe" in comment
  tracing: Let histogram values keep the percent and graph modifiers
  tracing: Keep the entry count when the histogram stats allocation fails
  tracing: Free histogram the field rejected for a bad modifier
  tracing: Free histogram the var ref when its initialization fails
  ...

1 files changed, 300 insertions, 0 deletions

diff --git a/Documentation/doc-guide/contributing.rst b/Documentation/doc-guide/contributing.rst
new file mode 100644
index 000000000..f8047e633
--- /dev/null
+++ b/Documentation/doc-guide/contributing.rst
@@ -0,0 +1,300 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+How to help improve kernel documentation
+========================================
+
+Documentation is an important part of any software-development project.
+Good documentation helps to bring new developers in and helps established
+developers work more effectively.  Without top-quality documentation, a lot
+of time is wasted in reverse-engineering the code and making avoidable
+mistakes.
+
+Unfortunately, the kernel's documentation currently falls far short of what
+it needs to be to support a project of this size and importance.
+
+This guide is for contributors who would like to improve that situation.
+Kernel documentation improvements can be made by developers at a variety of
+skill levels; they are a relatively easy way to learn the kernel process in
+general and find a place in the community.  The bulk of what follows is the
+documentation maintainer's list of tasks that most urgently need to be
+done.
+
+The documentation TODO list
+---------------------------
+
+There is an endless list of tasks that need to be carried out to get our
+documentation to where it should be.  This list contains a number of
+important items, but is far from exhaustive; if you see a different way to
+improve the documentation, please do not hold back!
+
+Addressing warnings
+~~~~~~~~~~~~~~~~~~~
+
+The documentation build currently spews out an unbelievable number of
+warnings.  When you have that many, you might as well have none at all;
+people ignore them, and they will never notice when their work adds new
+ones.  For this reason, eliminating warnings is one of the highest-priority
+tasks on the documentation TODO list.  The task itself is reasonably
+straightforward, but it must be approached in the right way to be
+successful.
+
+Warnings issued by a compiler for C code can often be dismissed as false
+positives, leading to patches aimed at simply shutting the compiler up.
+Warnings from the documentation build almost always point at a real
+problem; making those warnings go away requires understanding the problem
+and fixing it at its source.  For this reason, patches fixing documentation
+warnings should probably not say "fix a warning" in the changelog title;
+they should indicate the real problem that has been fixed.
+
+Another important point is that documentation warnings are often created by
+problems in kerneldoc comments in C code.  While the documentation
+maintainer appreciates being copied on fixes for these warnings, the
+documentation tree is often not the right one to actually carry those
+fixes; they should go to the maintainer of the subsystem in question.
+
+For example, in a documentation build I grabbed a pair of warnings nearly
+at random::
+
+  ./drivers/devfreq/devfreq.c:1818: warning: bad line:
+  	- Resource-managed devfreq_register_notifier()
+  ./drivers/devfreq/devfreq.c:1854: warning: bad line:
+	- Resource-managed devfreq_unregister_notifier()
+
+(The lines were split for readability).
+
+A quick look at the source file named above turned up a couple of kerneldoc
+comments that look like this::
+
+  /**
+   * devm_devfreq_register_notifier()
+	  - Resource-managed devfreq_register_notifier()
+   * @dev:	The devfreq user device. (parent of devfreq)
+   * @devfreq:	The devfreq object.
+   * @nb:		The notifier block to be unregistered.
+   * @list:	DEVFREQ_TRANSITION_NOTIFIER.
+   */
+
+The problem is the missing "*", which confuses the build system's
+simplistic idea of what C comment blocks look like.  This problem had been
+present since that comment was added in 2016 — a full four years.  Fixing
+it was a matter of adding the missing asterisks.  A quick look at the
+history for that file showed what the normal format for subject lines is,
+and ``scripts/get_maintainer.pl`` told me who should receive it (pass paths to
+your patches as arguments to scripts/get_maintainer.pl).  The resulting patch
+looked like this::
+
+  [PATCH] PM / devfreq: Fix two malformed kerneldoc comments
+
+  Two kerneldoc comments in devfreq.c fail to adhere to the required format,
+  resulting in these doc-build warnings:
+
+    ./drivers/devfreq/devfreq.c:1818: warning: bad line:
+  	  - Resource-managed devfreq_register_notifier()
+    ./drivers/devfreq/devfreq.c:1854: warning: bad line:
+	  - Resource-managed devfreq_unregister_notifier()
+
+  Add a couple of missing asterisks and make kerneldoc a little happier.
+
+  Signed-off-by: Jonathan Corbet <corbet@lwn.net>
+  ---
+   drivers/devfreq/devfreq.c | 4 ++--
+   1 file changed, 2 insertions(+), 2 deletions(-)
+
+  diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
+  index 57f6944d65a6..00c9b80b3d33 100644
+  --- a/drivers/devfreq/devfreq.c
+  +++ b/drivers/devfreq/devfreq.c
+  @@ -1814,7 +1814,7 @@ static void devm_devfreq_notifier_release(struct device *dev, void *res)
+
+   /**
+    * devm_devfreq_register_notifier()
+  -	- Resource-managed devfreq_register_notifier()
+  + *	- Resource-managed devfreq_register_notifier()
+    * @dev:	The devfreq user device. (parent of devfreq)
+    * @devfreq:	The devfreq object.
+    * @nb:		The notifier block to be unregistered.
+  @@ -1850,7 +1850,7 @@ EXPORT_SYMBOL(devm_devfreq_register_notifier);
+
+   /**
+    * devm_devfreq_unregister_notifier()
+  -	- Resource-managed devfreq_unregister_notifier()
+  + *	- Resource-managed devfreq_unregister_notifier()
+    * @dev:	The devfreq user device. (parent of devfreq)
+    * @devfreq:	The devfreq object.
+    * @nb:		The notifier block to be unregistered.
+  --
+  2.24.1
+
+The entire process only took a few minutes.  Of course, I then found that
+somebody else had fixed it in a separate tree, highlighting another lesson:
+always check linux-next to see if a problem has been fixed before you dig
+into it.
+
+Other fixes will take longer, especially those relating to structure
+members or function parameters that lack documentation.  In such cases, it
+is necessary to work out what the role of those members or parameters is
+and describe them correctly.  Overall, this task gets a little tedious at
+times, but it's highly important.  If we can actually eliminate warnings
+from the documentation build, then we can start expecting developers to
+avoid adding new ones.
+
+In addition to warnings from the regular documentation build, you can also
+run ``make refcheckdocs`` to find references to nonexistent documentation
+files.
+
+Languishing kerneldoc comments
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Developers are encouraged to write kerneldoc comments for their code, but
+many of those comments are never pulled into the docs build.  That makes
+this information harder to find and, for example, makes Sphinx unable to
+generate links to that documentation.  Adding ``kernel-doc`` directives to
+the documentation to bring those comments in can help the community derive
+the full value of the work that has gone into creating them.
+
+The ``tools/docs/find-unused-docs.sh`` tool can be used to find these
+overlooked comments.
+
+Note that the most value comes from pulling in the documentation for
+exported functions and data structures.  Many subsystems also have
+kerneldoc comments for internal use; those should not be pulled into the
+documentation build unless they are placed in a document that is
+specifically aimed at developers working within the relevant subsystem.
+
+
+Typo fixes
+~~~~~~~~~~
+
+Fixing typographical or formatting errors in the documentation is a quick
+way to figure out how to create and send patches, and it is a useful
+service.  I am always willing to accept such patches.  That said, once you
+have fixed a few, please consider moving on to more advanced tasks, leaving
+some typos for the next beginner to address.
+
+Please note that some things are *not* typos and should not be "fixed":
+
+ - Both American and British English spellings are allowed within the
+   kernel documentation.  There is no need to fix one by replacing it with
+   the other.
+
+ - The question of whether a period should be followed by one or two spaces
+   is not to be debated in the context of kernel documentation.  Other
+   areas of rational disagreement, such as the "Oxford comma", are also
+   off-topic here.
+
+As with any patch to any project, please consider whether your change is
+really making things better.
+
+Ancient documentation
+~~~~~~~~~~~~~~~~~~~~~
+
+Some kernel documentation is current, maintained, and useful.  Some
+documentation is ... not.  Dusty, old, and inaccurate documentation can
+mislead readers and casts doubt on our documentation as a whole.  Anything
+that can be done to address such problems is more than welcome.
+
+Whenever you are working with a document, please consider whether it is
+current, whether it needs updating, or whether it should perhaps be removed
+altogether.  There are a number of warning signs that you can pay attention
+to here:
+
+ - References to 2.x kernels
+ - Pointers to SourceForge repositories
+ - Nothing but typo fixes in the history for several years
+ - Discussion of pre-Git workflows
+
+The best thing to do, of course, would be to bring the documentation
+current, adding whatever information is needed.  Such work often requires
+the cooperation of developers familiar with the subsystem in question, of
+course.  Developers are often more than willing to cooperate with people
+working to improve the documentation when asked nicely, and when their
+answers are listened to and acted upon.
+
+Some documentation is beyond hope; we occasionally find documents that
+refer to code that was removed from the kernel long ago, for example.
+There is surprising resistance to removing obsolete documentation, but we
+should do that anyway.  Extra cruft in our documentation helps nobody.
+
+In cases where there is perhaps some useful information in a badly outdated
+document, and you are unable to update it, the best thing to do may be to
+add a warning at the beginning.  The following text is recommended::
+
+  .. warning ::
+  	This document is outdated and in need of attention.  Please use
+	this information with caution, and please consider sending patches
+	to update it.
+
+That way, at least our long-suffering readers have been warned that the
+document may lead them astray.
+
+Documentation coherency
+~~~~~~~~~~~~~~~~~~~~~~~
+
+The old-timers around here will remember the Linux books that showed up on
+the shelves in the 1990s.  They were simply collections of documentation
+files scrounged from various locations on the net.  The books have (mostly)
+improved since then, but the kernel's documentation is still mostly built
+on that model.  It is thousands of files, almost each of which was written
+in isolation from all of the others.  We don't have a coherent body of
+kernel documentation; we have thousands of individual documents.
+
+We have been trying to improve the situation through the creation of
+a set of "books" that group documentation for specific readers.  These
+include:
+
+ - Documentation/admin-guide/index.rst
+ - Documentation/core-api/index.rst
+ - Documentation/driver-api/index.rst
+ - Documentation/userspace-api/index.rst
+
+As well as this book on documentation itself.
+
+Moving documents into the appropriate books is an important task and needs
+to continue.  There are a couple of challenges associated with this work,
+though.  Moving documentation files creates short-term pain for the people
+who work with those files; they are understandably unenthusiastic about
+such changes.  Usually the case can be made to move a document once; we
+really don't want to keep shifting them around, though.
+
+Even when all documents are in the right place, though, we have only
+managed to turn a big pile into a group of smaller piles.  The work of
+trying to knit all of those documents together into a single whole has not
+yet begun.  If you have bright ideas on how we could proceed on that front,
+we would be more than happy to hear them.
+
+Stylesheet improvements
+~~~~~~~~~~~~~~~~~~~~~~~
+
+With the adoption of Sphinx we have much nicer-looking HTML output than we
+once did.  But it could still use a lot of improvement; Donald Knuth and
+Edward Tufte would be unimpressed.  That requires tweaking our stylesheets
+to create more typographically sound, accessible, and readable output.
+
+Be warned: if you take on this task you are heading into classic bikeshed
+territory.  Expect a lot of opinions and discussion for even relatively
+obvious changes.  That is, alas, the nature of the world we live in.
+
+Non-LaTeX PDF build
+~~~~~~~~~~~~~~~~~~~
+
+This is a decidedly nontrivial task for somebody with a lot of time and
+Python skills.  The Sphinx toolchain is relatively small and well
+contained; it is easy to add to a development system.  But building PDF or
+EPUB output requires installing LaTeX, which is anything but small or well
+contained.  That would be a nice thing to eliminate.
+
+The original hope had been to use the rst2pdf tool (https://rst2pdf.org/)
+for PDF generation, but it turned out to not be up to the task.
+Development work on rst2pdf seems to have picked up again in recent times,
+though, which is a hopeful sign.  If a suitably motivated developer were to
+work with that project to make rst2pdf work with the kernel documentation
+build, the world would be eternally grateful.
+
+Write more documentation
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Naturally, there are massive parts of the kernel that are severely
+underdocumented.  If you have the knowledge to document a specific kernel
+subsystem and the desire to do so, please do not hesitate to do some
+writing and contribute the result to the kernel.  Untold numbers of kernel
+developers and users will thank you.