Merge tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-tracegrafted

Pull tracing fixes from Steven Rostedt:

 - Don't destroy user event fields when removal fails

   User event fields are destroyed before the event is removed from
   visibility. But that can fail leaving the still visible event with no
   fields. Move the destroying of the fields to after the event is
   successfully removed from visibility.

 - Initialize function graph state is fork before calling
   copy_exec_state()

   For non-CLONE_VM forks, copy_exec_state() allocates a new
   task_exec_state. If that allocation fails, ftrace_graph_exit_task()
   will free the tasks ret_stack pointer. Since that pointer is still
   using the parent's ret_stack, it mistakenly frees the parent's
   pointer too.

   Call ftrace_graph_init() on the task first which will NULL out the
   new tasks's ret_stack and if the copy fails, it will not free
   anything.

 - Remove FGRAPH_MAX_INDEX

   The macro FGRAPH_MAX_INDEX was added but never used. Remove it.

 - Save ent_size in function graph printing of nested functions

   The function graph tracer needs to look at the next event to see if
   the next event is the return of the current function entry. If it is,
   it prints a single line:

	ktime_get();

   Otherwise it prints it like a nested function:

	tick_nohz_irq_exit() {
	    ktime_get();
	    kcpustat_irq_exit();
	}

   In order to look at the next event, it must save the current event so
   that it has the information to print from it. It saves the event in
   the iterator descriptor called "ent". What it doesn't save is the
   ent_size of the event which is now used to know if the function graph
   arguments are to be printed. The peek doesn't save the size so the
   size used happens to be that of the size of the last event that was
   seen.

   Save the entry event size in the iterator descriptor so that the
   correct size is used.

 - Fix several errors with freeing data in the histogram code

   The histogram code had a lot of leaked or or incorrect accounting
   when failures happen. Correct them.

 - Fix histogram regression of .percent and .graph modifiers

   Up until 6.3 histogram values could have "percent" or "graph"
   modifiers that changed how they were printed. But a change that added
   restricting histograms values from being strings, stack traces and
   other modifiers inadvertently prevented them from using the percent
   and graph modifiers, which were legal use cases for values.

   Put back the percent and graph modifiers.

 - Fix various typos in the comments

 - Set the trace_clock before initializing a histogram with clock
   argument

   The histogram API allows the user to specific which trace clock to
   use via a "clock=" string. The histogram is set up first before the
   clock is checked. If the passed in clock is not valid, it exits
   without fully fixing up the histogram leaving it on the list and a
   use-after-free can trigger.

   Update the clock argument first and if it fails then exit gracefully
   before the histogram trigger is placed on any lists.

 - Restore :mod: trailer after parsing in ftrace_set_clr_event

   The function ftrace_set_clr_event() modifies the parse string and
   needs to put it back to what was passed in. It searches for ":mod:"
   via a strsep() but fails to put back the first ':' in the string.

   Add back the ':' in the passed in string.

 - Take trace_array reference when opening a tracer options file

   The options files are dynamically created and some tracers add their
   own options. When a tracer adds their own list of options, the
   trace_array holding them has an array to hold the list of options for
   each tracer. This array increases in size via a krealloc(), and the
   new entry gets a newly allocated array to hold the options of the new
   tracer being added.

   The element in each entry of the tracer's option array holds a
   pointer back to the trace_array, a pointer to the tracer it is
   associated to, a pointer to the flags of the option.

   The issue is that these arrays are freed when the trace_array is
   freed when its instance it represents is removed from the instances
   directory. There's a race that an open of one of these options files
   can happen when the instance is being removed.

   Add a new helper function to be called by the open function of the
   options file to iterate all existing trace_arrays under a lock and
   find the one that has the given option element in one of it's tracer
   arrays. If found, then update the associated trace_array's reference
   counter to keep it from being freed. If not found, have the open call
   return -ENODEV.

 - Disable interrupts when acquiring the lock in rb_wake_up_waiters()

   The function rb_wake_up_waiters() assumes it will be called in
   interrupt context and does not disable irqs when taking
   cpu_buffer->reader_lock, which can be called in hard interrupt
   context. The issue is in PREEMPT_RT, this function is called in
   thread context leaving this lock open to a deadlock.

   Take the lock with interrupts disabled.

 - Use rcu_assign_pointer() for tmp_ops filter hash

   The tmp_ops used in update_ftrace_direct_mod() assigns its
   filter_hash field directly, but that field is annotated as __rcu and
   sparse complains. Assign it with rcu_assign_pointer()

 - Fix use-after-free in enable_trigger_private_data_free()

   The trace_event_call is accessed through the event_trigger_data's
   trace_event_file pointer to put the trace_event_call on freeing. The
   issue is that the trace_event_file data may have been freed already
   causing a use-after-free. Add a field to the event_trigger_data that
   points directly to the trace_event_call so that it can decrement its
   reference directly without needing to go through the
   trace_event_file.

 - Fix accounting of buffer data remote headers

   trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount
   the number of pages is needed for the asked for size as it doesn't
   take into account the meta data on each page. Add a helper function
   to do the calculation properly and use that in these functions.

 - Catch nr_page_va overflow in ring_buffer_desc sizing

   The number of pages per remote ring buffer is capped by
   ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
   overflow that field would silently allocate a descriptor smaller than
   what was asked for.

 - Do not resize the subbuf order if any per_cpu buffer is disabled

   The mmapping of ring buffers disables resizing the subbuffers, but it
   is done per-cpu whereas the subbuf size change is done for all the
   per_cpu buffers under the buffer->mutex. It could change the size of
   some while the mapping is happening on others. Have the resize of the
   subbuf order check all the per_cpu buffers under the lock to see if
   any of them is disabled before starting and causing an inconsistency
   between buffers that are being mapped.

* tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (25 commits)
  ring-buffer: Check resize_disabled before publishing the new subbuf order
  tracing/remotes: Catch nr_page_va overflow in ring_buffer_desc sizing
  tracing/remotes: Account for ring buffer page header in size calculation
  tracing: Don't dereference trace_event_file in deferred trigger free
  ftrace: Use rcu_assign_pointer() for tmp_ops filter hash
  ring-buffer: Acquire the lock with irqsave in rb_wake_up_waiters()
  tracing: Take trace_array reference when opening a tracer options file
  tracing: Fix ring_buffer_read_page_size() kernel-doc
  tracing: Restore :mod: trailer after parsing in ftrace_set_clr_event()
  tracing: Fix memory corruption from a "STACKTRACE" histogram key
  tracing: Fix memory corruption from the histogram stacktrace modifier
  tracing: Undo the registration when enabling the histogram trigger fails
  tracing: Take the reference before publishing the named histogram trigger
  tracing: Set the trace clock before registering the histogram trigger
  tracing: Fix typo "preceeded" in comment
  tracing: Fix typo "availabe" in comment
  tracing: Let histogram values keep the percent and graph modifiers
  tracing: Keep the entry count when the histogram stats allocation fails
  tracing: Free histogram the field rejected for a bad modifier
  tracing: Free histogram the var ref when its initialization fails
  ...

1 files changed, 514 insertions, 0 deletions

diff --git a/include/scsi/libiscsi.h b/include/scsi/libiscsi.h
new file mode 100644
index 000000000..3d765c77b
--- /dev/null
+++ b/include/scsi/libiscsi.h
@@ -0,0 +1,514 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * iSCSI lib definitions
+ *
+ * Copyright (C) 2006 Red Hat, Inc.  All rights reserved.
+ * Copyright (C) 2004 - 2006 Mike Christie
+ * Copyright (C) 2004 - 2005 Dmitry Yusupov
+ * Copyright (C) 2004 - 2005 Alex Aizman
+ */
+#ifndef LIBISCSI_H
+#define LIBISCSI_H
+
+#include <linux/types.h>
+#include <linux/wait.h>
+#include <linux/mutex.h>
+#include <linux/timer.h>
+#include <linux/workqueue.h>
+#include <linux/kfifo.h>
+#include <linux/refcount.h>
+#include <scsi/iscsi_proto.h>
+#include <scsi/iscsi_if.h>
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_transport_iscsi.h>
+
+struct scsi_transport_template;
+struct scsi_host_template;
+struct scsi_device;
+struct Scsi_Host;
+struct scsi_target;
+struct scsi_cmnd;
+struct socket;
+struct iscsi_transport;
+struct iscsi_cls_session;
+struct iscsi_cls_conn;
+struct iscsi_session;
+struct iscsi_nopin;
+struct device;
+
+#define ISCSI_DEF_XMIT_CMDS_MAX	128	/* must be power of 2 */
+#define ISCSI_MGMT_CMDS_MAX	15
+
+#define ISCSI_DEF_CMD_PER_LUN	32
+
+/* Task Mgmt states */
+enum {
+	TMF_INITIAL,
+	TMF_QUEUED,
+	TMF_SUCCESS,
+	TMF_FAILED,
+	TMF_TIMEDOUT,
+	TMF_NOT_FOUND,
+};
+
+#define ISID_SIZE			6
+
+/* Connection flags */
+#define ISCSI_CONN_FLAG_SUSPEND_TX	0
+#define ISCSI_CONN_FLAG_SUSPEND_RX	1
+#define ISCSI_CONN_FLAG_BOUND		2
+
+#define ISCSI_ITT_MASK			0x1fff
+#define ISCSI_TOTAL_CMDS_MAX		4096
+/* this must be a power of two greater than ISCSI_MGMT_CMDS_MAX */
+#define ISCSI_TOTAL_CMDS_MIN		16
+#define ISCSI_AGE_SHIFT			28
+#define ISCSI_AGE_MASK			0xf
+
+#define ISCSI_ADDRESS_BUF_LEN		64
+
+enum {
+	/* this is the maximum possible storage for AHSs */
+	ISCSI_MAX_AHS_SIZE = sizeof(struct iscsi_ecdb_ahdr) +
+				sizeof(struct iscsi_rlength_ahdr),
+	ISCSI_DIGEST_SIZE = sizeof(__u32),
+};
+
+
+enum {
+	ISCSI_TASK_FREE,
+	ISCSI_TASK_COMPLETED,
+	ISCSI_TASK_PENDING,
+	ISCSI_TASK_RUNNING,
+	ISCSI_TASK_ABRT_TMF,		/* aborted due to TMF */
+	ISCSI_TASK_ABRT_SESS_RECOV,	/* aborted due to session recovery */
+	ISCSI_TASK_REQUEUE_SCSIQ,	/* qcmd requeueing to scsi-ml */
+};
+
+struct iscsi_r2t_info {
+	__be32			ttt;		/* copied from R2T */
+	__be32			exp_statsn;	/* copied from R2T */
+	uint32_t		data_length;	/* copied from R2T */
+	uint32_t		data_offset;	/* copied from R2T */
+	int			data_count;	/* DATA-Out payload progress */
+	int			datasn;
+	/* LLDs should set/update these values */
+	int			sent;		/* R2T sequence progress */
+};
+
+struct iscsi_task {
+	/*
+	 * Because LLDs allocate their hdr differently, this is a pointer
+	 * and length to that storage. It must be setup at session
+	 * creation time.
+	 */
+	struct iscsi_hdr	*hdr;
+	unsigned short		hdr_max;
+	unsigned short		hdr_len;	/* accumulated size of hdr used */
+	/* copied values in case we need to send tmfs */
+	itt_t			hdr_itt;
+	__be32			cmdsn;
+	struct scsi_lun		lun;
+
+	int			itt;		/* this ITT */
+
+	unsigned		imm_count;	/* imm-data (bytes)   */
+	/* offset in unsolicited stream (bytes); */
+	struct iscsi_r2t_info	unsol_r2t;
+	char			*data;		/* mgmt payload */
+	unsigned		data_count;
+	struct scsi_cmnd	*sc;		/* associated SCSI cmd*/
+	struct iscsi_conn	*conn;		/* used connection    */
+
+	/* data processing tracking */
+	unsigned long		last_xfer;
+	unsigned long		last_timeout;
+	bool			have_checked_conn;
+
+	/* T10 protection information */
+	bool			protected;
+
+	/* state set/tested under session->lock */
+	int			state;
+	refcount_t		refcount;
+	struct list_head	running;	/* running cmd list */
+	void			*dd_data;	/* driver/transport data */
+};
+
+static inline int iscsi_task_has_unsol_data(struct iscsi_task *task)
+{
+	return task->unsol_r2t.data_length > task->unsol_r2t.sent;
+}
+
+static inline void* iscsi_next_hdr(struct iscsi_task *task)
+{
+	return (void*)task->hdr + task->hdr_len;
+}
+
+static inline bool iscsi_task_is_completed(struct iscsi_task *task)
+{
+	return task->state == ISCSI_TASK_COMPLETED ||
+	       task->state == ISCSI_TASK_ABRT_TMF ||
+	       task->state == ISCSI_TASK_ABRT_SESS_RECOV;
+}
+
+/* Private data associated with struct scsi_cmnd. */
+struct iscsi_cmd {
+	struct iscsi_task	*task;
+	int			age;
+};
+
+static inline struct iscsi_cmd *iscsi_cmd(struct scsi_cmnd *cmd)
+{
+	return scsi_cmd_priv(cmd);
+}
+
+/* Connection's states */
+enum {
+	ISCSI_CONN_INITIAL_STAGE,
+	ISCSI_CONN_STARTED,
+	ISCSI_CONN_STOPPED,
+	ISCSI_CONN_CLEANUP_WAIT,
+};
+
+struct iscsi_conn {
+	struct iscsi_cls_conn	*cls_conn;	/* ptr to class connection */
+	void			*dd_data;	/* iscsi_transport data */
+	struct iscsi_session	*session;	/* parent session */
+	/*
+	 * conn_stop() flag: stop to recover, stop to terminate
+	 */
+        int			stop_stage;
+	struct timer_list	transport_timer;
+	unsigned long		last_recv;
+	unsigned long		last_ping;
+	int			ping_timeout;
+	int			recv_timeout;
+	struct iscsi_task 	*ping_task;
+
+	/* iSCSI connection-wide sequencing */
+	uint32_t		exp_statsn;
+	uint32_t		statsn;
+
+	/* control data */
+	int			id;		/* CID */
+	int			c_stage;	/* connection state */
+	/*
+	 * Preallocated buffer for pdus that have data but do not
+	 * originate from scsi-ml. We never have two pdus using the
+	 * buffer at the same time. It is only allocated to
+	 * the default max recv size because the pdus we support
+	 * should always fit in this buffer
+	 */
+	char			*data;
+	struct iscsi_task 	*login_task;	/* mtask used for login/text */
+	struct iscsi_task	*task;		/* xmit task in progress */
+
+	/* xmit */
+	/* items must be added/deleted under frwd lock */
+	struct list_head	mgmtqueue;	/* mgmt (control) xmit queue */
+	struct list_head	cmdqueue;	/* data-path cmd queue */
+	struct list_head	requeue;	/* tasks needing another run */
+	struct work_struct	xmitwork;	/* per-conn. xmit workqueue */
+	/* recv */
+	struct work_struct	recvwork;
+	unsigned long		flags;		/* ISCSI_CONN_FLAGs */
+
+	/* negotiated params */
+	unsigned		max_recv_dlength; /* initiator_max_recv_dsl*/
+	unsigned		max_xmit_dlength; /* target_max_recv_dsl */
+	int			hdrdgst_en;
+	int			datadgst_en;
+	int			ifmarker_en;
+	int			ofmarker_en;
+	/* values userspace uses to id a conn */
+	int			persistent_port;
+	char			*persistent_address;
+
+	unsigned		max_segment_size;
+	unsigned		tcp_xmit_wsf;
+	unsigned		tcp_recv_wsf;
+	uint16_t		keepalive_tmo;
+	uint16_t		local_port;
+	uint8_t			tcp_timestamp_stat;
+	uint8_t			tcp_nagle_disable;
+	uint8_t			tcp_wsf_disable;
+	uint8_t			tcp_timer_scale;
+	uint8_t			tcp_timestamp_en;
+	uint8_t			fragment_disable;
+	uint8_t			ipv4_tos;
+	uint8_t			ipv6_traffic_class;
+	uint8_t			ipv6_flow_label;
+	uint8_t			is_fw_assigned_ipv6;
+	char			*local_ipaddr;
+
+	/* MIB-statistics */
+	uint64_t		txdata_octets;
+	uint64_t		rxdata_octets;
+	uint32_t		scsicmd_pdus_cnt;
+	uint32_t		dataout_pdus_cnt;
+	uint32_t		scsirsp_pdus_cnt;
+	uint32_t		datain_pdus_cnt;
+	uint32_t		r2t_pdus_cnt;
+	uint32_t		tmfcmd_pdus_cnt;
+	int32_t			tmfrsp_pdus_cnt;
+
+	/* custom statistics */
+	uint32_t		eh_abort_cnt;
+	uint32_t		fmr_unalign_cnt;
+};
+
+struct iscsi_pool {
+	struct kfifo		queue;		/* FIFO Queue */
+	void			**pool;		/* Pool of elements */
+	int			max;		/* Max number of elements */
+};
+
+/* Session's states */
+enum {
+	ISCSI_STATE_FREE = 1,
+	ISCSI_STATE_LOGGED_IN,
+	ISCSI_STATE_FAILED,
+	ISCSI_STATE_TERMINATE,
+	ISCSI_STATE_IN_RECOVERY,
+	ISCSI_STATE_RECOVERY_FAILED,
+	ISCSI_STATE_LOGGING_OUT,
+};
+
+struct iscsi_session {
+	struct iscsi_cls_session *cls_session;
+	/*
+	 * Syncs up the scsi eh thread with the iscsi eh thread when sending
+	 * task management functions. This must be taken before the session
+	 * and recv lock.
+	 */
+	struct mutex		eh_mutex;
+	/* abort */
+	wait_queue_head_t	ehwait;		/* used in eh_abort() */
+	struct iscsi_tm		tmhdr;
+	struct timer_list	tmf_timer;
+	int			tmf_state;	/* see TMF_INITIAL, etc.*/
+	struct iscsi_task	*running_aborted_task;
+
+	/* iSCSI session-wide sequencing */
+	uint32_t		cmdsn;
+	uint32_t		exp_cmdsn;
+	uint32_t		max_cmdsn;
+
+	/* This tracks the reqs queued into the initiator */
+	uint32_t		queued_cmdsn;
+
+	/* configuration */
+	int			abort_timeout;
+	int			lu_reset_timeout;
+	int			tgt_reset_timeout;
+	int			initial_r2t_en;
+	unsigned short		max_r2t;
+	int			imm_data_en;
+	unsigned		first_burst;
+	unsigned		max_burst;
+	int			time2wait;
+	int			time2retain;
+	int			pdu_inorder_en;
+	int			dataseq_inorder_en;
+	int			erl;
+	int			fast_abort;
+	int			tpgt;
+	char			*username;
+	char			*username_in;
+	char			*password;
+	char			*password_in;
+	char			*targetname;
+	char			*targetalias;
+	char			*ifacename;
+	char			*initiatorname;
+	char			*boot_root;
+	char			*boot_nic;
+	char			*boot_target;
+	char			*portal_type;
+	char			*discovery_parent_type;
+	uint16_t		discovery_parent_idx;
+	uint16_t		def_taskmgmt_tmo;
+	uint16_t		tsid;
+	uint8_t			auto_snd_tgt_disable;
+	uint8_t			discovery_sess;
+	uint8_t			chap_auth_en;
+	uint8_t			discovery_logout_en;
+	uint8_t			bidi_chap_en;
+	uint8_t			discovery_auth_optional;
+	uint8_t			isid[ISID_SIZE];
+
+	/* control data */
+	struct iscsi_transport	*tt;
+	struct Scsi_Host	*host;
+	struct iscsi_conn	*leadconn;	/* leading connection */
+	/* Between the forward and the backward locks exists a strict locking
+	 * hierarchy. The mutual exclusion zone protected by the forward lock
+	 * can enclose the mutual exclusion zone protected by the backward lock
+	 * but not vice versa.
+	 */
+	spinlock_t		frwd_lock;	/* protects session state, *
+						 * cmdsn, queued_cmdsn     *
+						 * session resources:      *
+						 * - cmdpool kfifo_out ,   *
+						 * - mgmtpool, queues	   */
+	spinlock_t		back_lock;	/* protects cmdsn_exp      *
+						 * cmdsn_max,              *
+						 * cmdpool kfifo_in        */
+	int			state;		/* session state           */
+	int			age;		/* counts session re-opens */
+
+	int			scsi_cmds_max; 	/* max scsi commands */
+	int			cmds_max;	/* size of cmds array */
+	struct iscsi_task	**cmds;		/* Original Cmds arr */
+	struct iscsi_pool	cmdpool;	/* PDU's pool */
+	void			*dd_data;	/* LLD private data */
+};
+
+enum {
+	ISCSI_HOST_SETUP,
+	ISCSI_HOST_REMOVED,
+};
+
+struct iscsi_host {
+	char			*initiatorname;
+	/* hw address or netdev iscsi connection is bound to */
+	char			*hwaddress;
+	char			*netdev;
+
+	wait_queue_head_t	session_removal_wq;
+	/* protects sessions and state */
+	spinlock_t		lock;
+	int			num_sessions;
+	int			state;
+
+	struct workqueue_struct	*workq;
+};
+
+/*
+ * scsi host template
+ */
+extern int iscsi_eh_abort(struct scsi_cmnd *sc);
+extern int iscsi_eh_recover_target(struct scsi_cmnd *sc);
+extern int iscsi_eh_session_reset(struct scsi_cmnd *sc);
+extern int iscsi_eh_device_reset(struct scsi_cmnd *sc);
+extern enum scsi_qc_status iscsi_queuecommand(struct Scsi_Host *host,
+					      struct scsi_cmnd *sc);
+extern enum scsi_timeout_action iscsi_eh_cmd_timed_out(struct scsi_cmnd *sc);
+
+/*
+ * iSCSI host helpers.
+ */
+#define iscsi_host_priv(_shost) \
+	(shost_priv(_shost) + sizeof(struct iscsi_host))
+
+extern int iscsi_host_set_param(struct Scsi_Host *shost,
+				enum iscsi_host_param param, char *buf,
+				int buflen);
+extern int iscsi_host_get_param(struct Scsi_Host *shost,
+				enum iscsi_host_param param, char *buf);
+extern int iscsi_host_add(struct Scsi_Host *shost, struct device *pdev);
+extern struct Scsi_Host *iscsi_host_alloc(const struct scsi_host_template *sht,
+					  int dd_data_size,
+					  bool xmit_can_sleep);
+extern void iscsi_host_remove(struct Scsi_Host *shost, bool is_shutdown);
+extern void iscsi_host_free(struct Scsi_Host *shost);
+extern int iscsi_target_alloc(struct scsi_target *starget);
+extern int iscsi_host_get_max_scsi_cmds(struct Scsi_Host *shost,
+					uint16_t requested_cmds_max);
+
+/*
+ * session management
+ */
+extern struct iscsi_cls_session *
+iscsi_session_setup(struct iscsi_transport *, struct Scsi_Host *shost,
+		    uint16_t, int, int, uint32_t, unsigned int);
+void iscsi_session_remove(struct iscsi_cls_session *cls_session);
+void iscsi_session_free(struct iscsi_cls_session *cls_session);
+extern void iscsi_session_teardown(struct iscsi_cls_session *);
+extern void iscsi_session_recovery_timedout(struct iscsi_cls_session *);
+extern int iscsi_set_param(struct iscsi_cls_conn *cls_conn,
+			   enum iscsi_param param, char *buf, int buflen);
+extern int iscsi_session_get_param(struct iscsi_cls_session *cls_session,
+				   enum iscsi_param param, char *buf);
+
+#define iscsi_session_printk(prefix, _sess, fmt, a...)	\
+	iscsi_cls_session_printk(prefix, _sess->cls_session, fmt, ##a)
+
+/*
+ * connection management
+ */
+extern struct iscsi_cls_conn *iscsi_conn_setup(struct iscsi_cls_session *,
+					       int, uint32_t);
+extern void iscsi_conn_teardown(struct iscsi_cls_conn *);
+extern int iscsi_conn_start(struct iscsi_cls_conn *);
+extern void iscsi_conn_stop(struct iscsi_cls_conn *, int);
+extern int iscsi_conn_bind(struct iscsi_cls_session *, struct iscsi_cls_conn *,
+			   int);
+extern void iscsi_conn_unbind(struct iscsi_cls_conn *cls_conn, bool is_active);
+extern void iscsi_conn_failure(struct iscsi_conn *conn, enum iscsi_err err);
+extern void iscsi_session_failure(struct iscsi_session *session,
+				  enum iscsi_err err);
+extern int iscsi_conn_get_param(struct iscsi_cls_conn *cls_conn,
+				enum iscsi_param param, char *buf);
+extern int iscsi_conn_get_addr_param(struct sockaddr_storage *addr,
+				     enum iscsi_param param, char *buf);
+extern void iscsi_suspend_tx(struct iscsi_conn *conn);
+extern void iscsi_suspend_rx(struct iscsi_conn *conn);
+extern void iscsi_suspend_queue(struct iscsi_conn *conn);
+extern void iscsi_conn_queue_xmit(struct iscsi_conn *conn);
+extern void iscsi_conn_queue_recv(struct iscsi_conn *conn);
+
+#define iscsi_conn_printk(prefix, _c, fmt, a...) \
+	iscsi_cls_conn_printk(prefix, ((struct iscsi_conn *)_c)->cls_conn, \
+			      fmt, ##a)
+
+/*
+ * pdu and task processing
+ */
+extern void iscsi_update_cmdsn(struct iscsi_session *, struct iscsi_nopin *);
+extern void iscsi_prep_data_out_pdu(struct iscsi_task *task,
+				    struct iscsi_r2t_info *r2t,
+				    struct iscsi_data *hdr);
+extern int iscsi_conn_send_pdu(struct iscsi_cls_conn *, struct iscsi_hdr *,
+				char *, uint32_t);
+extern int iscsi_complete_pdu(struct iscsi_conn *, struct iscsi_hdr *,
+			      char *, int);
+extern int __iscsi_complete_pdu(struct iscsi_conn *, struct iscsi_hdr *,
+				char *, int);
+extern int iscsi_verify_itt(struct iscsi_conn *, itt_t);
+extern struct iscsi_task *iscsi_itt_to_ctask(struct iscsi_conn *, itt_t);
+extern struct iscsi_task *iscsi_itt_to_task(struct iscsi_conn *, itt_t);
+extern void iscsi_requeue_task(struct iscsi_task *task);
+extern void iscsi_put_task(struct iscsi_task *task);
+extern void __iscsi_put_task(struct iscsi_task *task);
+extern bool iscsi_get_task(struct iscsi_task *task);
+extern void iscsi_complete_scsi_task(struct iscsi_task *task,
+				     uint32_t exp_cmdsn, uint32_t max_cmdsn);
+
+/*
+ * generic helpers
+ */
+extern void iscsi_pool_free(struct iscsi_pool *);
+extern int iscsi_pool_init(struct iscsi_pool *, int, void ***, int);
+extern int iscsi_switch_str_param(char **, char *);
+
+/*
+ * inline functions to deal with padding.
+ */
+static inline unsigned int
+iscsi_padded(unsigned int len)
+{
+	return (len + ISCSI_PAD_LEN - 1) & ~(ISCSI_PAD_LEN - 1);
+}
+
+static inline unsigned int
+iscsi_padding(unsigned int len)
+{
+	len &= (ISCSI_PAD_LEN - 1);
+	if (len)
+		len = ISCSI_PAD_LEN - len;
+	return len;
+}
+
+#endif