diff options
Diffstat (limited to 'docs/features')
| -rw-r--r-- | docs/features/passthrough.rst | 10 | ||||
| -rw-r--r-- | docs/features/responsestreaming.rst | 68 | ||||
| -rw-r--r-- | docs/features/streaming.rst | 102 | ||||
| -rw-r--r-- | docs/features/tcpproxy.rst | 31 |
4 files changed, 107 insertions, 104 deletions
diff --git a/docs/features/passthrough.rst b/docs/features/passthrough.rst index d68a49a9..91fcb9b6 100644 --- a/docs/features/passthrough.rst +++ b/docs/features/passthrough.rst @@ -13,10 +13,10 @@ mechanism: away. Note that mitmproxy's "Limit" option is often the better alternative here, as it is not affected by the limitations listed below. -If you want to peek into (SSL-protected) non-HTTP connections, check out the :ref:`tcpproxy` +If you want to peek into (SSL-protected) non-HTTP connections, check out the :ref:`tcp_proxy` feature. If you want to ignore traffic from mitmproxy's processing because of large response bodies, -take a look at the :ref:`responsestreaming` feature. +take a look at the :ref:`streaming` feature. How it works ------------ @@ -38,7 +38,7 @@ There are two important quirks to consider: - **In transparent mode, the ignore pattern is matched against the IP and ClientHello SNI host.** While we usually infer the hostname from the Host header if the ``--host`` argument is passed to mitmproxy, we do not have access to this information before the SSL handshake. If the client uses SNI however, then we treat the SNI host as an ignore target. -- **In regular mode, explicit HTTP requests are never ignored.** [#explicithttp]_ The ignore pattern is +- **In regular and upstream proxy mode, explicit HTTP requests are never ignored.** [#explicithttp]_ The ignore pattern is applied on CONNECT requests, which initiate HTTPS or clear-text WebSocket connections. Tutorial @@ -88,8 +88,8 @@ Here are some other examples for ignore patterns: .. seealso:: - - :ref:`tcpproxy` - - :ref:`responsestreaming` + - :ref:`tcp_proxy` + - :ref:`streaming` - mitmproxy's "Limit" feature .. rubric:: Footnotes diff --git a/docs/features/responsestreaming.rst b/docs/features/responsestreaming.rst deleted file mode 100644 index 6fa93271..00000000 --- a/docs/features/responsestreaming.rst +++ /dev/null @@ -1,68 +0,0 @@ -.. _responsestreaming: - -Response Streaming -================== - -By using mitmproxy's streaming feature, response contents can be passed to the client incrementally -before they have been fully received by the proxy. This is especially useful for large binary files -such as videos, where buffering the whole file slows down the client's browser. - -By default, mitmproxy will read the entire response, perform any indicated -manipulations on it and then send the (possibly modified) response to -the client. In some cases this is undesirable and you may wish to "stream" -the response back to the client. When streaming is enabled, the response is -not buffered on the proxy but directly sent back to the client instead. - -On the command-line -------------------- - -Streaming can be enabled on the command line for all response bodies exceeding a certain size. -The SIZE argument understands k/m/g suffixes, e.g. 3m for 3 megabytes. - -================== ================= -command-line ``--stream SIZE`` -================== ================= - -.. warning:: - - When response streaming is enabled, **streamed response contents will not be - recorded or preserved in any way.** - -.. note:: - - When response streaming is enabled, the response body cannot be modified by the usual means. - -Customizing Response Streaming ------------------------------- - -You can also use a script to customize exactly which responses are streamed. - -Responses that should be tagged for streaming by setting their ``.stream`` -attribute to ``True``: - -.. literalinclude:: ../../examples/complex/stream.py - :caption: examples/complex/stream.py - :language: python - -Implementation Details ----------------------- - -When response streaming is enabled, portions of the code which would have otherwise performed -changes on the response body will see an empty response body. Any modifications will be ignored. - -Streamed responses are usually sent in chunks of 4096 bytes. If the response is sent with a -``Transfer-Encoding: chunked`` header, the response will be streamed one chunk at a time. - -Modifying streamed data ------------------------ - -If the ``.stream`` attribute is callable, ``.stream`` will wrap the generator that yields all -chunks. - -.. literalinclude:: ../../examples/complex/stream_modify.py - :caption: examples/complex/stream_modify.py - :language: python - -.. seealso:: - - - :ref:`passthrough` diff --git a/docs/features/streaming.rst b/docs/features/streaming.rst new file mode 100644 index 00000000..82510843 --- /dev/null +++ b/docs/features/streaming.rst @@ -0,0 +1,102 @@ +.. _streaming: + +HTTP Streaming +============== + +By default, mitmproxy will read the entire request/response, perform any indicated +manipulations on it and then send the (possibly modified) message to +the other party. In some cases this is undesirable and you may wish to "stream" +the request/response. When streaming is enabled, the request/response is +not buffered on the proxy but directly sent to the server/client instead. +HTTP headers are still fully buffered before being sent. + +Request Streaming +----------------- + +Request streaming can be used to incrementally stream a request body to the server +before it has been fully received by the proxy. This is useful for large file uploads. + +Response Streaming +------------------ + +By using mitmproxy's streaming feature, response contents can be passed to the client incrementally +before they have been fully received by the proxy. This is especially useful for large binary files +such as videos, where buffering the whole file slows down the client's browser. + +On the command-line +------------------- + +Streaming can be enabled on the command line for all request and response bodies exceeding a certain size. +The SIZE argument understands k/m/g suffixes, e.g. 3m for 3 megabytes. + +================== ================= +command-line ``--set stream_large_bodies=SIZE`` +================== ================= + +.. warning:: + + When streaming is enabled, **streamed request/response contents will not be + recorded or preserved in any way.** + +.. note:: + + When streaming is enabled, the request/response body cannot be modified by the usual means. + +Customizing Streaming +--------------------- + +You can also use a script to customize exactly which requests or responses are streamed. + +Requests/Responses that should be tagged for streaming by setting their ``.stream`` +attribute to ``True``: + +.. literalinclude:: ../../examples/complex/stream.py + :caption: examples/complex/stream.py + :language: python + +Implementation Details +---------------------- + +When response streaming is enabled, portions of the code which would have otherwise performed +changes on the request/response body will see an empty body. Any modifications will be ignored. + +Streamed bodies are usually sent in chunks of 4096 bytes. If the response is sent with a +``Transfer-Encoding: chunked`` header, the response will be streamed one chunk at a time. + +Modifying streamed data +----------------------- + +If the ``.stream`` attribute is callable, ``.stream`` will wrap the generator that yields all +chunks. + +.. literalinclude:: ../../examples/complex/stream_modify.py + :caption: examples/complex/stream_modify.py + :language: python + +WebSocket Streaming +=================== + +The WebSocket streaming feature can be used to send the frames as soon as they arrive. This can be useful for large binary file transfers. + +On the command-line +------------------- + +Streaming can be enabled on the command line for all WebSocket frames + +================== ================= +command-line ``--set stream_websockets=true`` +================== ================= + +.. note:: + + When Web Socket streaming is enabled, the message payload cannot be modified. + +Implementation Details +---------------------- +When WebSocket streaming is enabled, portions of the code which may perform changes to the WebSocket message payloads will not have +any effect on the actual payload sent to the server as the frames are immediately forwarded to the server. +In contrast to HTTP streaming, where the body is not stored, the message payload will still be stored in the WebSocket Flow. + +.. seealso:: + + - :ref:`passthrough` diff --git a/docs/features/tcpproxy.rst b/docs/features/tcpproxy.rst deleted file mode 100644 index 0825c024..00000000 --- a/docs/features/tcpproxy.rst +++ /dev/null @@ -1,31 +0,0 @@ -.. _tcpproxy: - -TCP Proxy -========= - -In case mitmproxy does not handle a specific protocol, you can exempt -hostnames from processing, so that mitmproxy acts as a generic TCP forwarder. -This feature is closely related to the :ref:`passthrough` functionality, -but differs in two important aspects: - -- The raw TCP messages are printed to the event log. -- SSL connections will be intercepted. - -Please note that message interception or modification are not possible yet. -If you are not interested in the raw TCP messages, you should use the ignore domains feature. - -How it works ------------- - -================== ====================== -command-line ``--tcp HOST`` -mitmproxy shortcut :kbd:`O` then :kbd:`T` -================== ====================== - -For a detailed description how the hostname pattern works, please look at the :ref:`passthrough` -feature. - -.. seealso:: - - - :ref:`passthrough` - - :ref:`responsestreaming` |