| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
Under pipelining, requests were given a 5x timeout factor due to the
way that the timeout clock works in libcurl. Under CDN load,
connections were not being torn down quickly and it was only
this timer that led to disconnect and retry. So, we want to
break a connection that isn't making progress but that isn't
immediately possible. We'll compromise with a 60S timeout that
(we hope) will be neither too long for stalled connections nor
too short for large asset transfer requests.
|
|
|
|
specify extra components of a format string that aren't populated in snprintf
|
|
Incorporate the new libcurl 7.38.0 build with curl bug 1420
workaround. Add developer-centric testing code to evaluate
the workaround or a future fix for 1420.
|
|
of used handles and a fast handle factory that's thread-
correct.
|
|
reproduce data corruption via timeouts.
|
|
as transfers can appear delayed with deep pipelining and more
requests in the pool. Added bad HTTP status error (typically
getting a 0 back as HTTP status from libcurl) to the list of
retryable errors. There's a response stream problem with libcurl
and pipelining that induces this problem. Retrying helps but
may not be entirely safe. Watch bug 1420 on the libcurl sourceforge
bug tracker. Extend options of test/example program to include
un-ranged requests. Document the excessive data transfer induced
when ranged requests are disabled. This is an abnormal mode for
very rare users so we'll just eat that for now.
|
|
some problems disabling pipelining on a multi handle with outstanding
requests so build a more conservative system that allows requests
to drain before setting curl multi options. Would rather not have
this but it is significantly safer. "HttpPipelining" debug setting
is now fully dynamic. Connection limits can also be made dynamic
in the near future. Upped the default connection count back to 8 for
now but will revisit this in the tuning phase. It might be time to
combine mesh and textures into a single asset class. For normal
server operations that would be a clear path, but for server under
load, the current scheme may be better. Minor cleanup in logging
to elminate some redundant strings. Might add some more tracing to the
stall logic 'just in case'.
|
|
|
|
|
|
|
|
Start using DNS cache in legacy LLCurl code. Go to 15 seconds
particularly as we're using threaded resolver at this point.
Documentation cleanup. Add libcurl status checking and logging
for curl_easy_setopt() operations that fail. Shouldn't happen
and we'll just continue anyway but there's info in the logs to
track these down now. Cleaned up logic around FASTTIMER enable
defines used to evaluate pipeline stalls in main thread.
Removed long-standing thread race around caps strings and
URL construction. Not a significant risk but refactoring the
code to get rid of them removed one huge eyesore. It can be
made even slicker if desired (see notes).
|
|
Much improved. Unified the global and class options into a single
option list. Implemented static and dynamic setting paths as much
as possible. Dynamic path does require packet/RPC but otherwise
there's near unification. Dynamic modes can't get values back yet
due to the response/notifier scheme but this doesn't bother me.
Flatten global and class options into simpler struct-like entities.
Setter/getter available on these when needed (external APIs) but code
can otherwise fiddle directly when it knows what to do. Much duplicated
options/state removed from HttpPolicy. Comments cleaned up. Threads
better described and consistently mentioned in API docs. Integration
test extended for 503 responses with Reply-After headers.
|
|
termination test. Sheesh. Also get some more numbers out of the
example/load test program which drives traffic. This should
give some useful insights into how the current http throttle
works (or doesn't) with varying client demands.
|
|
Mesh repo is using three policy classes now: one for
large objects, one for GetMesh2 regions, one for
GetMesh regions. It's also detecting the presence
of the cap and using the correct class. Class
initialization cleaned up significantly in llappcorehttp
using data-directed code. Pulled in the changes to
HttpHeader done for sunshine-internal then did a
refactoring pass on the header callback which now
uses a unified approach to clean up and deliver
header information to all interested parties. Added
support for using Retry-After header information on
503 retries.
|
|
|
|
|
|
Not certain what the source of the short data is with one resident but I'm
going to make these problems retryable as they are transport-related. Lift
the retry detection into a method that should be reusable by others interested
in determining what is retryable. Trace output handling on the libcurl debug
callback was attrocious. Some unsafe length handling on my part was protected
by a second layer of defense. Made that correct and more useful by logging
actual data sizes during trace.
|
|
Content-Range
Don't rely on a response body being present should a
Content-Range header be parsed. Unit tests captured
the original crash and confirm the fix.
|
|
handle duplication code. Reviewed by Kelly
|
|
Header container moves from a vector of raw lines to a vector
of string pairs representing name/value pairs in headers. For
incoming headers, we normalize the name to lowercase and trim
it. Values are only left-trimmed. Outgoing headers are left
as-is. Simple find() method for the common case, forward and
reverse iterators for those few who need to do it themselves.
The HTTP status line (e.g. 'HTTP/1.1 200 Ok') is no longer treated
as a header to be returned to caller. Unit tests, as usual,
were a bear but they absolutely ensured outgoing HTTP header
conformance after the change. Grunt work paid off.
LLTextureFetch was also given a second options structure
for texture fetches. Same as the original but with header return
to caller requested. Baked textures should use this, the other
20,000 texture fetch requests should continue to use the original.
|
|
Added second mesh class as well as an asset upload class.
Refactored initialization to use less code and more data to
cleanly get http started. Modified mesh to use the new
http class for large requests (>2MB for now). Added additional
timeout setting to llcorehttp to distinguish connection timeout
from transport timeout and are now using transport timeout
values for large asset downloads that may need more time.
|
|
Header container moves from a vector of raw lines to a vector
of string pairs representing name/value pairs in headers. For
incoming headers, we normalize the name to lowercase and trim
it. Values are only left-trimmed. Outgoing headers are left
as-is. Simple find() method for the common case, forward and
reverse iterators for those few who need to do it themselves.
The HTTP status line (e.g. 'HTTP/1.1 200 Ok') is no longer treated
as a header to be returned to caller. Unit tests, as usual,
were a bear but they absolutely ensured outgoing HTTP header
conformance after the change. Grunt work paid off.
LLTextureFetch was also given a second options structure
for texture fetches. Same as the original but with header return
to caller requested. Baked textures should use this, the other
20,000 texture fetch requests should continue to use the original.
|
|
|
|
failures
|
|
changes to common libraries from the server codebase:
* Additional error checking in http handlers.
* Uniform log spam for http errors.
* Switch to using constants for http heads and status codes.
* Fixed bugs in incorrectly checking if parsing LLSD xml resulted in an error.
* Reduced spam regarding LLSD parsing errors in the default completedRaw http handler. It should not longer be necessary to short-circuit completedRaw to avoid spam.
* Ported over a few bug fixes from the server code.
* Switch mode http status codes to use S32 instead of U32.
* Ported LLSD::asStringRef from server code; avoids copying strings all over the place.
* Ported server change to LLSD::asBinary; this always returns a reference now instead of copying the entire binary blob.
* Ported server pretty notation format (and pretty binary format) to llsd serialization.
* The new LLCurl::Responder API no longer has two error handlers to choose from. Overriding the following methods have been deprecated:
** error - use httpFailure
** errorWithContent - use httpFailure
** result - use httpSuccess
** completed - use httpCompleted
** completedHeader - no longer necessary; call getResponseHeaders() from a completion method to obtain these headers.
* In order to 'catch' a completed http request, override one of these methods:
** httpSuccess - Called for any 2xx status code.
** httpFailure - Called for any non-2xx status code.
** httpComplete - Called for all status codes. Default implementation is to call either httpSuccess or httpFailure.
* It is recommended to keep these methods protected/private in order to avoid triggering of these methods without using a 'push' method (see below).
* Uniform error handling should followed whenever possible by calling a variant of this during httpFailure:
** llwarns << dumpResponse() << llendl;
* Be sure to include LOG_CLASS(your_class_name) in your class in order for the log entry to give more context.
* In order to 'push' a result into the responder, you should no longer call error, errorWithContent, result, or completed.
* Nor should you directly call httpSuccess/Failure/Completed (unless passing a message up to a parent class).
* Instead, you can set the internal content of a responder and trigger a corresponding method using the following methods:
** successResult - Sets results and calls httpSuccess
** failureResult - Sets results and calls httpFailure
** completedResult - Sets results and calls httpCompleted
* To obtain information about a the response from a reponder method, use the following getters:
** getStatus - HTTP status code
** getReason - Reason string
** getContent - Content (Parsed body LLSD)
** getResponseHeaders - Response Headers (LLSD map)
** getHTTPMethod - HTTP method of the request
** getURL - URL of the request
* It is still possible to override completeRaw if you want to manipulate data directly out of LLPumpIO.
* See indra/llmessage/llcurl.h for more information.
|
|
|
|
request. During readcallback, would generate an overrun-type message
about reading position beyond end-of-data. Mistake was is messaging
when state is exactly at end of data (which is expected) versus an
overrun. Both result in declaring end-of-data to libcurl. Also
changed some of the status logging for the metrics payload to be
less chatty on success, more informative on error.
|
|
Add to-do list to _httpinternal.h to guide anyone who
wants to pitch in and help.
|
|
Define expectations for headers for GET, POST, PUT requests.
Document those in the interface, test those with integration tests.
Verify that header overrides work as expected.
|
|
When releasing HTTP waiters, avoid unnecessary sort activity.
For Content-Type in responses, let libcurl do the work and removed
my parsing of headers. Drop Content-Encoding as libcurl will deal
with that. If anyone is interested, they can parse.
|
|
HttpResponse object now has two strings for these content headers.
Either or both may be empty. Tidied up the cross-platform string
code and got more defensive about the length of a header line.
Integration test for the new response object.
|
|
Also added some comments and changed the callback userdata argument
to be an HttpOpRequest rather than a libcurl handle. Less code,
less clutter.
|
|
Think I have found the major factor that causes the Linksys WRT54G V5 to
fall over in testing scenarios: DNS. For some historical reason, we're
trying to use libcurl without any DNS caching. My implementation echoed
that and implemented it correctly and I was seeing a DNS request per request
on the wire. The existing implementation tries to do that and has bugs
because it is clearing caching DNS data querying only once every few
seconds. Once I started emulating the bug, comms through the WRT became
much, much more reliable.
|
|
Only thing interesting in this changeset is the discovery that a sleep
in the fake HTTP server ties up tests. Need to thread that or fail on
client disconnect or something to speed that up and make it usable for
bigger test scenarios. But good enough for now...
|
|
<offset, length, fulllength>.
|
|
|
|
Pretty straightforward. Still don't like how I'm managing
the options block. Struct? Accessors? Can't decide. But
the options now speed up the unit test runs even as I add
tests.
|
|
LLProxy support, HttpOptions starting to work, HTTP resource waiting fixed.
Non-LLThread-based threads need to do some registration or LLMutex locks taken out in these
threads will not work as expected (SH-3154). We'll get a better solution later, this fixes
some things for now. Tracing of operations now supported. Global and per-request (via
HttpOptions) tracing levels of [0..3]. The 2 and 3 levels use libcurl's VERBOSE mode
combined with CURLOPT_DEBUGFUNCTION to stream high levels of detail into the log. *Very*
laggy but useful. Simple GET request supported (no Range: header). Really just a
degenrate case of a ranged get but supplied an API anyway. Global option to use the
LLProxy interface to setup CURL handles for either socks5 or http proxy usage. This
isn't really the most encapsulated way to do this but a better solution will have to
come later. The wantHeaders and tracing options are now supported in HttpOptions giving
per-request controls. Big refactoring of the HTTP resource waiter in lltexturefetch.
What I was doing before wasn't correct. Instead, I'm implementing the resource wait
after the Semaphore model (though not using system semaphores). So instead of having
a sequence like: SEND_HTTP_REQ -> WAIT_HTTP_RESOURCE -> SEND_HTTP_REQ, we now
do WAIT_HTTP_RESOURCE -> WAIT_HTTP_RESOURCE2 (actual wait) -> SEND_HTTP_REQ. Works
well but the prioritized filling of the corehttp library needs some performance
work later.
|
|
Implemented/modified PUT & POST to not used chunked encoding for the request.
Made the unit test much happier and probably a better thing for the pipeline.
Have a cheesy static & dynamic proxy capability using both local options and
a way to wire into LLProxy in llmessages. Not a clean thing but it will get
the proxy path working with both socks5 & http proxies. Refactoring to get
rid of unneeded library handler and unified an HttpStatus return for all
requests. Big batch of code removed as a result of that and more is possible
as well as some syscall avoidance with a bit more work. Boosted the unit
tests for simple PUT & POST test which revealed the test harness does *not*
like chunked encoding so we'll avoid it for now (and don't really need it
in any of our schemes).
|
|
206/content-range hack in xport.
Retry/response handling is decided in policy so moved that there. Removed special case
206-without-content-range response in transport. Have this sitation recognizable in the
API and let callers deal with it as needed.
|
|
surprised me. Added a retry queue similar to ready queue to the
policy object which is sorted by retry time. Currently do five
retries (after the initial try) delayed by .25, .5, 1, 2 and 5
seconds. Removed the retry logic from the lltexturefetch module.
Upped the waiting time in the unit test for the retries. People
won't like this but tough, need tests.
|
|
Implemented first global policy definitions to support SSL CA certificate configuration
to support https: operations. Fixed HTTP 206 status handling to match what is currently
being done by grid services and to lay a foundation for fixes that will be a response
to ER-1824. More libcurl CURLOPT options set on easy handles to do peer verification
in the traditional way. HTTP POST working and now reporting asset metrics back to
grid for the viewer's asset system. This uses LLSD so that is also showing as compatible
with the new library.
|
|
1 byte of data.
Shouldn't be making that kind of mistake.
|
|
chunking data. Remove the stateful use of a seek pointer so
that shared read is possible (though maybe not interesting).
|
|
excuse to go through an use a typedef for priority and policy class id.
|
|
Identified and reacted to the priority inversion problem we
have in texturefetch. Includes the introduction of a priority_queue
for the requests that are ready. Start some parameterization in
anticipation of having policy_class everywhere. Removed _assert.h
which isn't really needed in indra codebase. Implemented async
setPriority request (which I hope I can get rid of eventually along
with all priorities in this library). Converted to using unsigned
int for priority rather than float. Implemented POST and did
groundwork for PUT.
|
|
This is the first functional viewer pass with the HTTP work of the texture fetch
code performed by the llcorehttp library. Not exactly a 'drop-in' replacement
but a work-alike with some changes (e.g. handler notification in consumer
thread versus responder notification in worker thread).
This also includes some temporary changes in the priority scheme to prevent
the kind of priority inversion found in VWR-28996. Scheme used here does
provide liveness if not optimal responsiveness or order-of-operation.
The llcorehttp library at this point is far from optimally performing.
Its worker thread is making relatively poor use of cycles it gets and
it doesn't idle or sleep intelligently yet. This early integration step
helps shake out the interfaces, implementation niceties will be covered
soon.
|
