drm/amd/display: Try RGB before YCbCr 4:4:4 in stream validationgrafted

amdgpu_dm_create_validate_stream_for_sink() walks encoding_order[] and
uses the first encoding that validates. YCbCr 4:4:4 is listed before
RGB, so an HDMI sink that advertises 4:4:4 gets YCbCr 4:4:4 whenever the
"color format" property is left at AUTO, even though RGB fits the same
link.

That contradicts the documented AUTO behaviour for HDMI in enum
drm_connector_color_format (RGB, falling back to YCbCr 4:2:0 only when
the bandwidth is not available or the mode is 4:2:0-only), which the
amdgpu implementation of the property also describes. It also leaves
the "Broadcast RGB" property without effect on such sinks, since the
quantization range it selects only applies to RGB output.

Try RGB first. The mask still holds every encoding the sink supports,
so a mode that cannot carry RGB falls back exactly as before.

For reference, v7.2 picked RGB here unless YCbCr 4:4:4 was forced
through debugfs, while earlier kernels picked YCbCr 4:4:4 for any HDMI
sink that advertised it.

Fixes: 0b0ff65d3ca1 ("drm/amd/display: Refactor stream validation")
Suggested-by: Adolfo Rodrigues <adolfotregosa@gmail.com>
Assisted-by: Claude Code:claude-fable-5-1
Signed-off-by: Adrian Betschart <adrian.betschart@cinemaone.ch>
Reviewed-by: Fangzhi Zuo <jerry.zuo@amd.com>
Tested-by: Adolfo Rodrigues <adolfotregosa@gmail.com>
Tested-by: Daniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 7de432bc753133764dc40d37f9388da56de251e7)

5 files changed, 599 insertions, 0 deletions

diff --git a/Documentation/driver-api/80211/cfg80211.rst b/Documentation/driver-api/80211/cfg80211.rst
new file mode 100644
index 000000000..836f609c3
--- /dev/null
+++ b/Documentation/driver-api/80211/cfg80211.rst
@@ -0,0 +1,178 @@
+==================
+cfg80211 subsystem
+==================
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: Introduction
+
+Device registration
+===================
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: Device registration
+
+.. kernel-doc:: include/net/cfg80211.h
+   :functions:
+	ieee80211_channel_flags
+	ieee80211_channel
+	ieee80211_rate_flags
+	ieee80211_rate
+	ieee80211_sta_ht_cap
+	ieee80211_supported_band
+	cfg80211_signal_type
+	wiphy_params_flags
+	wiphy_flags
+	wiphy
+	wireless_dev
+	wiphy_new
+	wiphy_read_of_freq_limits
+	wiphy_register
+	wiphy_unregister
+	wiphy_free
+	wiphy_name
+	wiphy_dev
+	wiphy_priv
+	priv_to_wiphy
+	set_wiphy_dev
+	wdev_priv
+	ieee80211_iface_limit
+	ieee80211_iface_combination
+	cfg80211_check_combinations
+
+Actions and configuration
+=========================
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: Actions and configuration
+
+.. kernel-doc:: include/net/cfg80211.h
+   :functions:
+	cfg80211_ops
+	vif_params
+	key_params
+	survey_info_flags
+	survey_info
+	cfg80211_beacon_data
+	cfg80211_ap_settings
+	station_parameters
+	rate_info_flags
+	rate_info
+	station_info
+	monitor_flags
+	mpath_info_flags
+	mpath_info
+	bss_parameters
+	ieee80211_txq_params
+	cfg80211_crypto_settings
+	cfg80211_auth_request
+	cfg80211_assoc_request
+	cfg80211_deauth_request
+	cfg80211_disassoc_request
+	cfg80211_ibss_params
+	cfg80211_connect_params
+	cfg80211_pmksa
+	cfg80211_rx_mlme_mgmt
+	cfg80211_auth_timeout
+	cfg80211_rx_assoc_resp
+	cfg80211_assoc_timeout
+	cfg80211_tx_mlme_mgmt
+	cfg80211_ibss_joined
+	cfg80211_connect_resp_params
+	cfg80211_connect_done
+	cfg80211_connect_result
+	cfg80211_connect_bss
+	cfg80211_connect_timeout
+	cfg80211_roamed
+	cfg80211_disconnected
+	cfg80211_ready_on_channel
+	cfg80211_remain_on_channel_expired
+	cfg80211_new_sta
+	cfg80211_rx_mgmt
+	cfg80211_mgmt_tx_status
+	cfg80211_cqm_rssi_notify
+	cfg80211_cqm_pktloss_notify
+	cfg80211_michael_mic_failure
+
+Scanning and BSS list handling
+==============================
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: Scanning and BSS list handling
+
+.. kernel-doc:: include/net/cfg80211.h
+   :functions:
+	cfg80211_ssid
+	cfg80211_scan_request
+	cfg80211_scan_done
+	cfg80211_bss
+	cfg80211_inform_bss
+	cfg80211_inform_bss_frame_data
+	cfg80211_inform_bss_data
+	cfg80211_unlink_bss
+	cfg80211_find_ie
+	ieee80211_bss_get_ie
+
+Utility functions
+=================
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: Utility functions
+
+.. kernel-doc:: include/net/cfg80211.h
+   :functions:
+	ieee80211_channel_to_frequency
+	ieee80211_frequency_to_channel
+	ieee80211_get_channel
+	ieee80211_get_response_rate
+	ieee80211_hdrlen
+	ieee80211_get_hdrlen_from_skb
+	ieee80211_radiotap_iterator
+
+Data path helpers
+=================
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: Data path helpers
+
+.. kernel-doc:: include/net/cfg80211.h
+   :functions:
+	ieee80211_data_to_8023
+	ieee80211_amsdu_to_8023s
+	cfg80211_classify8021d
+
+Regulatory enforcement infrastructure
+=====================================
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: Regulatory enforcement infrastructure
+
+.. kernel-doc:: include/net/cfg80211.h
+   :functions:
+	regulatory_hint
+	wiphy_apply_custom_regulatory
+	freq_reg_info
+
+RFkill integration
+==================
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: RFkill integration
+
+.. kernel-doc:: include/net/cfg80211.h
+   :functions:
+	wiphy_rfkill_set_hw_state
+	wiphy_rfkill_start_polling
+	wiphy_rfkill_stop_polling
+
+Test mode
+=========
+
+.. kernel-doc:: include/net/cfg80211.h
+   :doc: Test mode
+
+.. kernel-doc:: include/net/cfg80211.h
+   :functions:
+	cfg80211_testmode_alloc_reply_skb
+	cfg80211_testmode_reply
+	cfg80211_testmode_alloc_event_skb
+	cfg80211_testmode_event
diff --git a/Documentation/driver-api/80211/index.rst b/Documentation/driver-api/80211/index.rst
new file mode 100644
index 000000000..62305e9c3
--- /dev/null
+++ b/Documentation/driver-api/80211/index.rst
@@ -0,0 +1,10 @@
+=====================================
+Linux 802.11 Driver Developer's Guide
+=====================================
+
+.. toctree::
+
+   introduction
+   cfg80211
+   mac80211
+   mac80211-advanced
diff --git a/Documentation/driver-api/80211/introduction.rst b/Documentation/driver-api/80211/introduction.rst
new file mode 100644
index 000000000..4938fa876
--- /dev/null
+++ b/Documentation/driver-api/80211/introduction.rst
@@ -0,0 +1,17 @@
+============
+Introduction
+============
+
+Explaining wireless 802.11 networking in the Linux kernel
+
+Copyright 2007-2009 Johannes Berg
+
+These books attempt to give a description of the various subsystems
+that play a role in 802.11 wireless networking in Linux. Since these
+books are for kernel developers they attempts to document the
+structures and functions used in the kernel as well as giving a
+higher-level overview.
+
+The reader is expected to be familiar with the 802.11 standard as
+published by the IEEE in 802.11-2007 (or possibly later versions).
+References to this standard will be given as "802.11-2007 8.1.5".
diff --git a/Documentation/driver-api/80211/mac80211-advanced.rst b/Documentation/driver-api/80211/mac80211-advanced.rst
new file mode 100644
index 000000000..f8df7b3af
--- /dev/null
+++ b/Documentation/driver-api/80211/mac80211-advanced.rst
@@ -0,0 +1,239 @@
+=============================
+mac80211 subsystem (advanced)
+=============================
+
+Information contained within this part of the book is of interest only
+for advanced interaction of mac80211 with drivers to exploit more
+hardware capabilities and improve performance.
+
+LED support
+===========
+
+Mac80211 supports various ways of blinking LEDs. Wherever possible,
+device LEDs should be exposed as LED class devices and hooked up to the
+appropriate trigger, which will then be triggered appropriately by
+mac80211.
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_get_tx_led_name
+	ieee80211_get_rx_led_name
+	ieee80211_get_assoc_led_name
+	ieee80211_get_radio_led_name
+	ieee80211_tpt_blink
+	ieee80211_tpt_led_trigger_flags
+	ieee80211_create_tpt_led_trigger
+
+Hardware crypto acceleration
+============================
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: Hardware crypto acceleration
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	set_key_cmd
+	ieee80211_key_conf
+	ieee80211_key_flags
+	ieee80211_get_tkip_p1k
+	ieee80211_get_tkip_p1k_iv
+	ieee80211_get_tkip_p2k
+
+Powersave support
+=================
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: Powersave support
+
+Beacon filter support
+=====================
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: Beacon filter support
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions: ieee80211_beacon_loss
+
+Multiple queues and QoS support
+===============================
+
+TBD
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions: ieee80211_tx_queue_params
+
+Access point mode support
+=========================
+
+TBD
+
+Some parts of the if_conf should be discussed here instead
+
+Insert notes about VLAN interfaces with hw crypto here or in the hw
+crypto chapter.
+
+support for powersaving clients
+-------------------------------
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: AP support for powersaving clients
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_get_buffered_bc
+	ieee80211_beacon_get
+	ieee80211_sta_eosp
+	ieee80211_frame_release_type
+	ieee80211_sta_ps_transition
+	ieee80211_sta_ps_transition_ni
+	ieee80211_sta_set_buffered
+	ieee80211_sta_block_awake
+
+Supporting multiple virtual interfaces
+======================================
+
+TBD
+
+Note: WDS with identical MAC address should almost always be OK
+
+Insert notes about having multiple virtual interfaces with different MAC
+addresses here, note which configurations are supported by mac80211, add
+notes about supporting hw crypto with it.
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_iterate_active_interfaces
+	ieee80211_iterate_active_interfaces_atomic
+
+Station handling
+================
+
+TODO
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_sta
+	sta_notify_cmd
+	ieee80211_find_sta
+	ieee80211_find_sta_by_ifaddr
+
+Hardware scan offload
+=====================
+
+TBD
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions: ieee80211_scan_completed
+
+Aggregation
+===========
+
+TX A-MPDU aggregation
+---------------------
+
+.. kernel-doc:: net/mac80211/agg-tx.c
+   :doc: TX A-MPDU aggregation
+
+.. WARNING: DOCPROC directive not supported: !Cnet/mac80211/agg-tx.c
+
+RX A-MPDU aggregation
+---------------------
+
+.. kernel-doc:: net/mac80211/agg-rx.c
+   :doc: RX A-MPDU aggregation
+
+.. WARNING: DOCPROC directive not supported: !Cnet/mac80211/agg-rx.c
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions: ieee80211_ampdu_mlme_action
+
+Spatial Multiplexing Powersave (SMPS)
+=====================================
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: Spatial multiplexing power save
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_request_smps
+	ieee80211_smps_mode
+
+TBD
+
+This part of the book describes the rate control algorithm interface and
+how it relates to mac80211 and drivers.
+
+Rate Control API
+================
+
+TBD
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_start_tx_ba_session
+	ieee80211_start_tx_ba_cb_irqsafe
+	ieee80211_stop_tx_ba_session
+	ieee80211_stop_tx_ba_cb_irqsafe
+	ieee80211_rate_control_changed
+	ieee80211_tx_rate_control
+
+TBD
+
+This part of the book describes mac80211 internals.
+
+Key handling
+============
+
+Key handling basics
+-------------------
+
+.. kernel-doc:: net/mac80211/key.c
+   :doc: Key handling basics
+
+MORE TBD
+--------
+
+TBD
+
+Receive processing
+==================
+
+TBD
+
+Transmit processing
+===================
+
+TBD
+
+Station info handling
+=====================
+
+Programming information
+-----------------------
+
+.. kernel-doc:: net/mac80211/sta_info.h
+   :functions:
+	sta_info
+	ieee80211_sta_info_flags
+
+STA information lifetime rules
+------------------------------
+
+.. kernel-doc:: net/mac80211/sta_info.c
+   :doc: STA information lifetime rules
+
+Aggregation Functions
+=====================
+
+.. kernel-doc:: net/mac80211/sta_info.h
+   :functions:
+	sta_ampdu_mlme
+	tid_ampdu_tx
+	tid_ampdu_rx
+
+Synchronisation Functions
+=========================
+
+TBD
+
+Locking, lots of RCU
diff --git a/Documentation/driver-api/80211/mac80211.rst b/Documentation/driver-api/80211/mac80211.rst
new file mode 100644
index 000000000..e38a22040
--- /dev/null
+++ b/Documentation/driver-api/80211/mac80211.rst
@@ -0,0 +1,155 @@
+===========================
+mac80211 subsystem (basics)
+===========================
+
+You should read and understand the information contained within this
+part of the book while implementing a mac80211 driver. In some chapters,
+advanced usage is noted, those may be skipped if this isn't needed.
+
+This part of the book only covers station and monitor mode
+functionality, additional information required to implement the other
+modes is covered in the second part of the book.
+
+Basic hardware handling
+=======================
+
+TBD
+
+This chapter shall contain information on getting a hw struct allocated
+and registered with mac80211.
+
+Since it is required to allocate rates/modes before registering a hw
+struct, this chapter shall also contain information on setting up the
+rate/mode structs.
+
+Additionally, some discussion about the callbacks and the general
+programming model should be in here, including the definition of
+ieee80211_ops which will be referred to a lot.
+
+Finally, a discussion of hardware capabilities should be done with
+references to other parts of the book.
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_hw
+	ieee80211_hw_flags
+	SET_IEEE80211_DEV
+	SET_IEEE80211_PERM_ADDR
+	ieee80211_ops
+	ieee80211_alloc_hw
+	ieee80211_register_hw
+	ieee80211_unregister_hw
+	ieee80211_free_hw
+
+PHY configuration
+=================
+
+TBD
+
+This chapter should describe PHY handling including start/stop callbacks
+and the various structures used.
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_conf
+	ieee80211_conf_flags
+
+Virtual interfaces
+==================
+
+TBD
+
+This chapter should describe virtual interface basics that are relevant
+to the driver (VLANs, MGMT etc are not.) It should explain the use of
+the add_iface/remove_iface callbacks as well as the interface
+configuration callbacks.
+
+Things related to AP mode should be discussed there.
+
+Things related to supporting multiple interfaces should be in the
+appropriate chapter, a BIG FAT note should be here about this though and
+the recommendation to allow only a single interface in STA mode at
+first!
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions: ieee80211_vif
+
+Receive and transmit processing
+===============================
+
+what should be here
+-------------------
+
+TBD
+
+This should describe the receive and transmit paths in mac80211/the
+drivers as well as transmit status handling.
+
+Frame format
+------------
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: Frame format
+
+Packet alignment
+----------------
+
+.. kernel-doc:: net/mac80211/rx.c
+   :doc: Packet alignment
+
+Calling into mac80211 from interrupts
+-------------------------------------
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: Calling mac80211 from interrupts
+
+functions/definitions
+---------------------
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_rx_status
+	mac80211_rx_encoding_flags
+	mac80211_rx_flags
+	mac80211_tx_info_flags
+	mac80211_tx_control_flags
+	mac80211_rate_control_flags
+	ieee80211_tx_rate
+	ieee80211_tx_info
+	ieee80211_tx_info_clear_status
+	ieee80211_rx
+	ieee80211_rx_ni
+	ieee80211_rx_irqsafe
+	ieee80211_tx_status_skb
+	ieee80211_tx_status_ni
+	ieee80211_tx_status_irqsafe
+	ieee80211_rts_get
+	ieee80211_rts_duration
+	ieee80211_ctstoself_get
+	ieee80211_ctstoself_duration
+	ieee80211_generic_frame_duration
+	ieee80211_wake_queue
+	ieee80211_stop_queue
+	ieee80211_wake_queues
+	ieee80211_stop_queues
+	ieee80211_queue_stopped
+
+Frame filtering
+===============
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: Frame filtering
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions: ieee80211_filter_flags
+
+The mac80211 workqueue
+======================
+
+.. kernel-doc:: include/net/mac80211.h
+   :doc: mac80211 workqueue
+
+.. kernel-doc:: include/net/mac80211.h
+   :functions:
+	ieee80211_queue_work
+	ieee80211_queue_delayed_work