You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- Instances of the client are unconditionally thread-safe.
100
100
101
101
## Prerequisites
@@ -725,13 +725,13 @@ print(response)
725
725
-[Cloudant blog](https://blog.cloudant.com/):
726
726
Many useful articles about how to optimize Cloudant for common problems.
727
727
728
-
### Changes feed follower (beta)
728
+
### Changes feed follower
729
729
730
730
#### Introduction
731
731
732
-
The SDK provides a changes feed follower utility (currently beta).
732
+
The SDK provides a changes feed follower utility.
733
733
This helper utility connects to the `_changes` endpoint and returns the individual change items.
734
-
It removes some of the complexity of using the `_changes` endpoint by setting some options automatically
734
+
It removes some complexity of using the `_changes` endpoint by setting some options automatically
735
735
and providing error suppression and retries.
736
736
737
737
*Tip: the changes feed often does not meet user expectations or assumptions.*
@@ -743,52 +743,52 @@ to get a better understanding of the limitations and suitable use-cases before u
743
743
744
744
There are two modes of operation:
745
745
* Start mode
746
-
* Fetches the changes from the supplied `since` sequence (by default the feed will start from `now`).
746
+
* Fetches the changes from the supplied `since` sequence (in this mode follower defaults to reading the feed from `now`).
747
747
* Fetches all available changes and then continues listening for new changes indefinitely unless encountering an end condition.
748
748
* An example use case for this mode is event driven workloads.
749
749
* Start one-off mode
750
-
* Fetches the changes from the supplied `since` sequence (by default the feed will start from the beginning).
750
+
* Fetches the changes from the supplied `since` sequence (in this mode follower defaults to reading the feed from the beginning).
751
751
* Fetches all available changes and then stops when either there are no further changes pending or encountering an end condition.
752
752
* An example use case for this mode is ETL style workloads.
753
753
754
754
#### Configuring the changes follower
755
755
756
756
The SDK's model of changes feed options is also used to configure the follower.
757
-
However, a subset of the options are invalid as they are configured internally by the implementation.
757
+
However, a subset of the options used internally by the follower implementation are invalid.
758
758
Supplying these options when instantiating the follower causes an error.
759
759
The invalid options are:
760
760
*`descending`
761
761
*`feed`
762
762
*`heartbeat`
763
763
*`lastEventId` - use `since` instead
764
764
*`timeout`
765
-
*Only the value of `_selector` is permitted for the `filter` option. This restriction is because selector
766
-
based filters perform better than JavaScript backed filters. Configuring a non-selector based filter will
767
-
cause the follower to error.
765
+
*Follower permits only the value `_selector` for the `filter` option. This restriction is because selector
766
+
based filters perform better than JavaScript backed filters. Configuring a non-selector based filter
767
+
causes the follower to error.
768
768
769
-
Note that the `limit` parameter will terminate the follower at the given number of changes in either
769
+
Note that the `limit` parameter terminates the follower at the given number of changes in either
770
770
operating mode.
771
771
772
-
The changes follower requires the client to have HTTP timeouts of at least 1 minute and will error during
772
+
The changes follower requires the client to have HTTP timeouts of at least 1 minute and errors during
773
773
instantiation if it is insufficient. The default client configuration has sufficiently long timeouts.
774
774
775
-
For use-cases where these configuration limitations are deemed too restrictive then it is recommended to
776
-
write code to use the SDK's [POST `_changes` API](https://github.com/IBM/cloudant-python-sdk/tree/v0.10.4/examples#postchanges) instead of the follower.
775
+
For use-cases where these configuration limitations are too restrictive then write code to use the SDK's
776
+
[POST `_changes` API](https://github.com/IBM/cloudant-python-sdk/tree/v0.10.4/examples#postchanges) instead of the follower.
777
777
778
778
#### Error suppression
779
779
780
-
By default, the changes follower will suppress transient errors indefinitely and attempt to run to completion or listen forever as
780
+
By default, the changes follower suppresses transient errors indefinitely and attempts to run to completion or listen forever as
781
781
dictated by the operating mode.
782
-
For applications where that is not desirable an optional error tolerance duration may be specified to control the time since
783
-
the last successful response that transient errors will be suppressed. This can be used, for example, by applications as a grace period
782
+
For applications where that is not desirable configure the optional error tolerance duration. This controls the time since
783
+
the last successful response that the follower suppresses transient errors. An example usage is an application grace period
784
784
before reporting an error and requiring intervention.
785
785
786
786
There are some additional points to consider for error suppression:
787
-
* Errors considered terminal, for example, the database not existing or invalid credentials are never suppressed and will error immediately.
788
-
* The error suppression duration is not guaranteed to fire immediately after lapsing and should be considered a minimum suppression time.
789
-
* The changes follower will back-off between retries and as such may remain paused for a short while after the transient errors have resolved.
790
-
* If the underlying SDK client used to initialize the follower also has retries configured then errors could be suppressed for significantly
791
-
longer than the follower's configured error tolerance duration depending on the configuration options.
787
+
* Errors considered terminal, for example, the database not existing or invalid credentials are never suppressed and error immediately.
788
+
* The error suppression duration is not guaranteed to fire immediately after lapsing and is a minimum suppression time.
789
+
* The changes follower backs-off between retries and as such may remain paused for a short while after the transient errors have resolved.
790
+
* If the underlying SDK client used to initialize the follower also has retries configured then suppression of errors may last
791
+
significantly longer than the follower's configured error tolerance duration depending on the specific options.
792
792
793
793
#### Follower operation
794
794
@@ -797,41 +797,41 @@ For both modes:
797
797
* A terminal error (HTTP codes `400`, `401`, `403``404`).
798
798
* Transient errors occur for longer than the error tolerance duration. Transient errors are all other HTTP status codes and connection errors.
799
799
* The number of changes received reaches the configured `limit`.
800
-
* The feed is terminated early by calling stop.
800
+
* The application calls stop to terminate the feed early.
801
801
802
-
As is true for the `_changes` endpoint change items have *at least once* delivery and an individual item
803
-
may be received multiple times. When using the follower change items may be repeated even within a limited
804
-
number of changes (i.e. using the `limit` option) this is a minor difference from using `limit` on the HTTP native API.
802
+
As is true for the `_changes` endpoint change items have *at least once* delivery and callers may receive
803
+
an individual item multiple times. When using the follower change items may repeat even within a limited
804
+
number of changes (that is using the `limit` option) this is a minor difference from using `limit` on the HTTP native API.
805
805
806
806
The follower is not optimized for some use cases and it is not recommended to use it in cases where:
807
807
* Setting `include_docs` and larger document sizes (for example > 10 kiB).
808
-
* The volume of changes is very high (if the rate of changes in the database exceeds the follower's rate of pulling them it will never catch-up).
808
+
* The volume of changes is very high (if the rate of changes in the database exceeds the follower's rate of pulling them it can never catch-up).
809
809
810
-
In these cases use-case specific control over the number of change requests made and the content size of the responses
811
-
may be achieved by using the SDK's [POST `_changes` API](https://github.com/IBM/cloudant-python-sdk/tree/v0.10.4/examples#postchanges).
810
+
In these use-cases usethe SDK's [POST `_changes` API](https://github.com/IBM/cloudant-python-sdk/tree/v0.10.4/examples#postchanges)
811
+
for specific control over the number of change requests made and the content size of the responses.
812
812
813
-
#### Checkpointing
813
+
#### Checkpoints
814
814
815
-
The changes follower does not checkpoint since it has no information about whether a change item has been
816
-
processed by the consuming application after being received. It is the application developer's responsibility
815
+
The changes follower does not checkpoint since it has no information about whether the consuming application
816
+
has processed a change item after delivery. It is the application developer's responsibility
817
817
to store the sequence IDs to have appropriate checkpoints and to re-initialize the follower with the required
818
818
`since` value after, for example, the application restarts.
819
819
820
-
The frequency and conditions for checkpointing are application specific and some applications may be tolerant
821
-
of dropped changes. This section is intended only to provide general guidance on how to avoid missing changes.
820
+
The frequency and conditions for checkpoints are application specific and some applications may be tolerant
821
+
of dropped changes. This section provides only general guidance on how to avoid missing changes.
822
822
823
-
To guarantee processing of all changes the sequence ID from a change item must not be persisted until *after*
824
-
the processing of the change item by the application has completed. As indicated previously change items are
825
-
delivered *at least once* so application code must be able to handle repeated changes already and it is
823
+
To guarantee processing of all changes do not persist the sequence ID from a change item until *after*
824
+
the processing of the change item by the application has completed. As indicated previously change item
825
+
delivery is *at least once* so application code must be able to handle repeated changes already. It is
826
826
preferable to restart from an older `since` value and receive changes again than risk missing them.
827
827
828
-
The sequence IDs are available on each change item by default, but may be omitted from some change items when
829
-
using the `seq_interval` configuration option. Infrequent sequence IDs may improve performance by reducing
830
-
the amount of data that needs to be transferred, but the trade-off is that more changes will be repeated if
831
-
it is necessary to resume the changes follower.
828
+
The sequence IDs are available on each change item by default. However, the server omits sequence IDs from
829
+
some change items when using the `seq_interval` configuration option.
830
+
Infrequent sequence IDs may improve performance by reducing the amount of data transfer and server load,
831
+
but the tradeoff is repeating more changes if it is necessary to resume the changes follower.
832
832
833
-
Extreme care should be taken with persisting sequences if choosing to process change items in parallel as there
834
-
is a considerable risk of missing changes on a restart if the sequence is recorded out of order.
833
+
Take extreme care persisting sequences if choosing to process change items in parallel as there
834
+
is a considerable risk of missing changes on a restart if the recorded sequence is out of order.
0 commit comments