-
Notifications
You must be signed in to change notification settings - Fork 162
Add v1/orders POST endpoint to get orders in batches #4048
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
c5928c0
e5d0199
b768ac8
f897dcd
319f5d1
f4be44b
0c7b052
6af9b50
01eccfe
9b5d618
35c2ce7
f26a539
508aa5b
cff1a94
17a26c3
3dbd193
7c11725
5de3e9c
c0dd2d9
e6d7bc9
373e9f1
de0bc19
67e8fc8
babc062
d8b161c
091c1f6
648e414
88cfce9
b450ab6
eeb9f86
9945081
73f61de
3a0377f
080bcd0
75ce8f0
9cfbf54
87a1b6e
44026f6
2cd718b
2bec199
7186254
c36972b
1799de4
fe9a493
c6fae97
fa734bf
9015c47
1edd7bc
c0284e1
b4f168f
28dbfe9
5dd9627
a20e537
df906c5
917dfe6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -731,7 +731,9 @@ pub struct OrderMetadata { | |||||||
| pub quote: Option<OrderQuote>, | ||||||||
| } | ||||||||
|
|
||||||||
| pub const ORDER_UID_LIMIT: usize = 1024; | ||||||||
| /// OrderUid is 56 bytes. When hex encoded as 0x prefixes Json string it is 116. | ||||||||
| /// Chosen to be under the MAX_JSON_BODY_PAYLOAD size of 1024 * 16 | ||||||||
| pub const ORDER_UID_LIMIT: usize = 128; | ||||||||
|
m-sz marked this conversation as resolved.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This constant is also used here:
1024 -> 128 is a breaking change.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is not truly a breaking change since the MAX_JSON_BODY_PAYLOAD was limiting the cancellation data order uids anyway to 139 (The amount of order uids that fit with the SignedOrderCancellations). I decided to reuse the same limiting constant for both requests and rounded it up to 128. If the 1024 (original) limit needs to be effective, We will need to increase the MAX_JSON_BODY_PAYLOAD, or do away with any order count limiting and just use MAX_JSON_BODY_PAYLOAD.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure I am following. From the main branch:
services/crates/model/src/order.rs Line 505 in 1c28e49
It is literally the maximum number of UIDs in the requests. It was 1024, and now it is 128. Did I miss anything?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, there is a default body limit at the api_router layer: api_router
.layer(DefaultBodyLimit::max(MAX_JSON_BODY_PAYLOAD as usize))which limits the request body to 16kb which equals to about 139 orders in the case of order cancellation payload. Which ultimately was the breaking change limiting the effective amount of order uids to 139.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Either we can also discuss removing the ORDER_UID_LIMIT altogether in both instances since we have the body limit in place.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah, thanks! I missed that. Does it mean that with this PR it will now be limited to 128 instead of 1024?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In this PR it will be limited to 128 instead of 139 effectively. The previous limit of 1024 was unrealistically high considering the size of hex encoded OrderUid and the body limit of 16kb.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. By the way, You can try it out for yourself as is currently: This will fail, the body is larger than 16kb (despite the 1024 max order limit, we are sending 133) 132 orders will not fail the size check:
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I mean, the code you share will work with 128 limit instead of 1024, no? Before this PR, the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. MAX_JSON_BODY_PAYLOAD was 16kb, and ORDER_UID_LIMIT was 1024 those are distinct. MAX_JSON_BODY_PAYLOAD effectively limits every request to every endpoint to be under 16kB, while ORDER_UID_LIMIT is a custom constant used today in cancellations, and with this PR in bulk order querying to limit amount of orders that can come in a request. The MAX_JSON_BODY_PAYLOAD or ORDER_UID_LIMIT both have a chance to fire at a specific request. If the request containing a large amount of orders exceeds the body payload limit, it will not even go into the handler. As discussed on Slack, let's keep both limits in place, since the ORDER_UID_LIMIT tells the user exactly what is wrong and what is the amount of orders that can be submitted at once. |
||||||||
|
|
||||||||
| // uid as 56 bytes: 32 for orderDigest, 20 for ownerAddress and 4 for validTo | ||||||||
| #[derive(Clone, Copy, Eq, Hash, PartialEq, PartialOrd, Ord)] | ||||||||
|
|
||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| use { | ||
| crate::api::AppState, | ||
| anyhow::Result, | ||
| axum::{ | ||
| extract::State, | ||
| http::StatusCode, | ||
| response::{IntoResponse, Response}, | ||
| }, | ||
| model::order::{ORDER_UID_LIMIT, Order, OrderUid}, | ||
| serde::Serialize, | ||
| std::sync::Arc, | ||
| }; | ||
|
|
||
| #[expect(clippy::large_enum_variant)] | ||
| #[derive(Debug, Serialize)] | ||
| #[serde(rename_all = "camelCase")] | ||
| enum OrderResultEntry { | ||
| Order(Order), | ||
| Error(OrderError), | ||
| } | ||
|
|
||
| #[derive(Debug, Serialize)] | ||
| #[serde(rename_all = "camelCase")] | ||
| struct OrderError { | ||
| uid: OrderUid, | ||
| description: String, | ||
| } | ||
|
|
||
| pub async fn get_orders_by_uid_handler( | ||
| State(state): State<Arc<AppState>>, | ||
| axum::Json(orders): axum::Json<Vec<OrderUid>>, | ||
|
m-sz marked this conversation as resolved.
|
||
| ) -> Response { | ||
| if orders.len() > ORDER_UID_LIMIT { | ||
| return ( | ||
| StatusCode::BAD_REQUEST, | ||
| format!("Request exceeds maximum number of order UIDs of {ORDER_UID_LIMIT}"), | ||
| ) | ||
| .into_response(); | ||
| } | ||
|
|
||
| get_orders_by_uid_response(state.orderbook.get_orders(&orders).await) | ||
| } | ||
|
|
||
| fn get_orders_by_uid_response(result: Result<Vec<(OrderUid, Result<Order>)>>) -> Response { | ||
| match result { | ||
| Ok(orders) => axum::Json( | ||
| orders | ||
| .into_iter() | ||
| .map(|(uid, order)| match order { | ||
| Ok(order) => OrderResultEntry::Order(order), | ||
| Err(err) => { | ||
| tracing::warn!(?err, "Error converting into model order"); | ||
|
m-sz marked this conversation as resolved.
|
||
| OrderResultEntry::Error(OrderError { | ||
| uid, | ||
| description: "Internal server error encountered when retrieving the \ | ||
| order" | ||
| .to_string(), | ||
| }) | ||
| } | ||
| }) | ||
| .collect::<Vec<OrderResultEntry>>(), | ||
| ) | ||
| .into_response(), | ||
| Err(err) => { | ||
| tracing::error!(?err, "get_orders_by_uid_response"); | ||
| crate::api::internal_error_reply() | ||
| } | ||
| } | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use {super::*, crate::api::response_body}; | ||
|
|
||
| #[tokio::test] | ||
| async fn get_orders_by_uid_ok() { | ||
| let order = Order::default(); | ||
| let uid = order.metadata.uid; | ||
| let result = vec![(uid, Ok(order.clone()))]; | ||
| let response = get_orders_by_uid_response(Ok(result)); | ||
| assert_eq!(response.status(), StatusCode::OK); | ||
|
|
||
| let body = response_body(response).await; | ||
| let entries: Vec<serde_json::Value> = serde_json::from_slice(&body).unwrap(); | ||
| assert_eq!(entries.len(), 1); | ||
|
|
||
| let order_entry: Order = | ||
| serde_json::from_value(entries[0].get("order").expect("key order exists").clone()) | ||
| .expect("value is a correct Order"); | ||
| assert_eq!(order_entry, order); | ||
| } | ||
|
|
||
| #[tokio::test] | ||
| async fn get_orders_by_uid_conversion_error() { | ||
| let uid = OrderUid([1u8; 56]); | ||
| let result = vec![(uid, Err(anyhow::anyhow!("bad data")))]; | ||
| let response = get_orders_by_uid_response(Ok(result)); | ||
| assert_eq!(response.status(), StatusCode::OK); | ||
|
|
||
| let body = response_body(response).await; | ||
| let entries: Vec<serde_json::Value> = serde_json::from_slice(&body).unwrap(); | ||
| assert_eq!(entries.len(), 1); | ||
|
|
||
| let error = entries[0].get("error").expect("error key exists"); | ||
| let error_uid: OrderUid = error.get("uid").unwrap().as_str().unwrap().parse().unwrap(); | ||
| assert_eq!(error_uid, uid); | ||
| assert_eq!( | ||
| error | ||
| .get("description") | ||
| .expect("key description exists") | ||
| .as_str() | ||
| .expect("description is a string"), | ||
| "Internal server error encountered when retrieving the order" | ||
| ); | ||
| } | ||
|
|
||
| #[tokio::test] | ||
| async fn get_orders_by_uid_err() { | ||
| let response = get_orders_by_uid_response(Err(anyhow::anyhow!("error"))); | ||
| assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR); | ||
|
m-sz marked this conversation as resolved.
|
||
| } | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.