summaryrefslogtreecommitdiff
path: root/indra/llcorehttp
AgeCommit message (Collapse)Author
2014-05-13sunshine-external merge WIPBrad Payne (Vir Linden)
2014-02-25merge viewer-release to sunshine-externalBrad Payne (Vir Linden)
2014-02-24merge with releaseRichard Linden
2014-01-09fixed unit tests being generated as Windows GUI projects instead of console ↵Richard Linden
projects
2013-10-16mergeBrad Payne (Vir Linden)
2013-10-08merge from viewer-releaseRichard Linden
2013-10-04Convert one more unit test over to improved waiting scheme to avoid build ↵Monty Brandenberg
failures.
2013-10-04Merge. Pull in viewer-release after 3.6.7 release.Monty Brandenberg
2013-09-27Up the transfer timeout of small meshes to 120S. This matchesMonty Brandenberg
the change made for MAINT-2347. Large transfers are still 10 minutes. Add/update to-do list and add some more info to the FAQ in the Readme.
2013-09-24SH-3690 SH-4505 Cleanup pass through code.Monty Brandenberg
Start using DNS cache in legacy LLCurl code. Go to 15 seconds particularly as we're using threaded resolver at this point. Documentation cleanup. Add libcurl status checking and logging for curl_easy_setopt() operations that fail. Shouldn't happen and we'll just continue anyway but there's info in the logs to track these down now. Cleaned up logic around FASTTIMER enable defines used to evaluate pipeline stalls in main thread. Removed long-standing thread race around caps strings and URL construction. Not a significant risk but refactoring the code to get rid of them removed one huge eyesore. It can be made even slicker if desired (see notes).
2013-09-18SH-4492 Create a useful README for llcorehttpMonty Brandenberg
Last bit for this release. Describe stream adapters and how to select a policy class. Slight changes to setup code to make reality reflect documentation.
2013-09-17SH-4492 Create a useful README for llcorehttp.Monty Brandenberg
First edit complete. Use the library in 15 minutes. Describe the code. Refinements to the initial try. Describe that code. Still to do: more refinements, how to choose a policy class, FAQ.
2013-09-11SH-4490 More 'humane' error code presentation from llcorehttp callersMonty Brandenberg
Added toTerseString() conversion on HttpStatus to generate a string that's more descriptive than the hex value of the HttpStatus value but still forms a short, searchable token (e.g. "Http_503" or "Core_7"). Using this throughout the viewer now, no live cases of toHex(), I believe.
2013-09-11SH-4489 New debug/dev settings for control over new mesh behaviorMonty Brandenberg
Added 'MeshUseGetMesh1' and 'MeshUseHttpRetryAfter' debug settings to control mesh transport behavior. First forces the use of the legacy mesh fetch style with high concurrency and connection churn. The second, on by default, honors Retry-After values if they are reasonable. If off or unreasonable, internal delay times are used.
2013-08-19Add 'internal'/'external' token to DEBUG retry message so that dev/QAMonty Brandenberg
can know exactly where a retry value was sourced.
2013-08-16SH-4407 Tuning to get new code working as well.Monty Brandenberg
Do some runtime code avoidance and skip unnecessary libcurl and syscall invocations.
2013-07-30SH-4371 Reduce 22mS inter-connection latency.Monty Brandenberg
This really extended into the client-side request throttling. Moved this from llmeshrepository (which doesn't really want to do connection management) into llcorehttp. It's now a class option with configurable rate. This still isn't the right thing to do as it creates coupling between viewer and services. When we get to pipelining, this notion becomes invalid.
2013-07-29Reduce HTTP request retry log spam. Thought I'd done this in anMonty Brandenberg
earlier maintenance branch but the code never showed up. I'll do it again. Spam is still available by bumping 'CoreHttp' tag up to DEBUGS level logging. Needed for QA. Can also get this data from tracing.
2013-07-29SH-4371 Reduce 22mS inter-conenction latency on HTTP operations.Monty Brandenberg
Simple change dropped this value by 7-10mS or so. Any time we complete an operation on a transport pass, arrange to skip the run-loop sleep so that we fill a possible empty slot as quickly as possible. With pipelining, this kind of thing should become unnecessary but for now, we'll do this to increase throughput.
2013-07-29SH-4368 Adjust upload timeout parameters for slow networks.Monty Brandenberg
Generally sorted the mesh timeout parameters for maximum transport time (staying with default 30 for connect). 60S for normal meshes, 600S for large. Also documented default option values in httpoptions.h. Useful to have these. In the future, the timeouts might go into standard llsd options where they can be tracked a bit more.
2013-07-12SH-4312 Configuration data between viewer and llcorehttp is clumsy.Monty Brandenberg
Much improved. Unified the global and class options into a single option list. Implemented static and dynamic setting paths as much as possible. Dynamic path does require packet/RPC but otherwise there's near unification. Dynamic modes can't get values back yet due to the response/notifier scheme but this doesn't bother me. Flatten global and class options into simpler struct-like entities. Setter/getter available on these when needed (external APIs) but code can otherwise fiddle directly when it knows what to do. Much duplicated options/state removed from HttpPolicy. Comments cleaned up. Threads better described and consistently mentioned in API docs. Integration test extended for 503 responses with Reply-After headers.
2013-07-08Found the memory corruptor. String trimmer didn't have a validMonty Brandenberg
termination test. Sheesh. Also get some more numbers out of the example/load test program which drives traffic. This should give some useful insights into how the current http throttle works (or doesn't) with varying client demands.
2013-07-03Update the example program so it handles meshes as well. Fix theMonty Brandenberg
request feed logic to use high/low-water level logic as is done in viewer code.
2013-06-27SH-4310/BUG-2810/MAINT-2794 Better status checking and error logging in ↵Monty Brandenberg
Mesh code. Pay correct attention to status codes coming back from services. Generate better and consistent error messages when problems arise. There's more to do in error handling, need a way to cleanly fail all request types, only have that for LOD at this point. Do better keeping the HTTP pipeline between the low and high water marks. This was made challenging because the outer most code couldn't really see what's going on internally (whose actions are delayed in a worker thread). More to do here, the debug-like requests don't honor limits, that will come later. Made retry counts available from llcorehttp which can be used by the throttle-anticipating logic to advance the count. It helps but it reinforces the coupling between viewer and server which I do not like.
2013-06-20SH-4257 Preparation for a new cap grant: GetMesh2Monty Brandenberg
Mesh repo is using three policy classes now: one for large objects, one for GetMesh2 regions, one for GetMesh regions. It's also detecting the presence of the cap and using the correct class. Class initialization cleaned up significantly in llappcorehttp using data-directed code. Pulled in the changes to HttpHeader done for sunshine-internal then did a refactoring pass on the header callback which now uses a unified approach to clean up and deliver header information to all interested parties. Added support for using Retry-After header information on 503 retries.
2013-06-05merge with viewer-releaseRichard Linden
2013-05-21mergeBrad Payne (Vir Linden)
2013-05-14SH-4190 Reactivate unit tests neutered under MAINT-3202Monty Brandenberg
HttpStatus unit tests have never caused a unit test failure but they do have the word 'error' in their text descriptions which gets picked up by the log processor in TC builds. So, reactivate the tests and reform the descriptions.
2013-05-14MAINT-2302 Tests frequently fail on Windows builds.Monty Brandenberg
I'm calling this the more correct fix. The httprequest tests actually talk to an HTTP server running in the Python test scaffold script. Under severe CPU competition, it may not get the cycles needed to start up and make progress. So this modifies the test to spin a little faster and dwell waiting on the server for a longer period. Hope this will be adequate to make the tests reliable.
2013-05-13Merge. Refresh from /lindenlab/viewer-cat prior to pushing.Monty Brandenberg
2013-05-13Whinge reduction effort on the unit tests. Python 2.7 SocketServerMonty Brandenberg
library has new exception-throwing behavior when a client disconnects unannounced. Generally ignore exceptions as a result as we don't care about the server side. On HTTP trace-mode tests, spin a little faster and longer to give libcurl time to emit all the junk it wants to send us. Should reduce 'reasonable time' failures on tests <12> and <13>.
2013-05-09Merge downstream code and viewer-betasimon
2013-05-06BUG-2295/MAINT-2624 unexpected crash around Content-Range: header processingMonty Brandenberg
Not certain what the source of the short data is with one resident but I'm going to make these problems retryable as they are transport-related. Lift the retry detection into a method that should be reusable by others interested in determining what is retryable. Trace output handling on the libcurl debug callback was attrocious. Some unsafe length handling on my part was protected by a second layer of defense. Made that correct and more useful by logging actual data sizes during trace.
2013-04-29BUG-2295/MAINT-2624 [FIXED] Crash in HttpOpRequest::stageFromActive w/ ↵Monty Brandenberg
Content-Range Don't rely on a response body being present should a Content-Range header be parsed. Unit tests captured the original crash and confirm the fix.
2013-04-25Revert ares and libcurl version update that was causing problems, revise curlsimon
handle duplication code. Reviewed by Kelly
2013-04-19merge up to latest viewer-development for merge to 3.5.2Oz Linden
2013-04-15SH-4106 Significantly upgrade the HttpHeaders interface for SSB.Monty Brandenberg
Header container moves from a vector of raw lines to a vector of string pairs representing name/value pairs in headers. For incoming headers, we normalize the name to lowercase and trim it. Values are only left-trimmed. Outgoing headers are left as-is. Simple find() method for the common case, forward and reverse iterators for those few who need to do it themselves. The HTTP status line (e.g. 'HTTP/1.1 200 Ok') is no longer treated as a header to be returned to caller. Unit tests, as usual, were a bear but they absolutely ensured outgoing HTTP header conformance after the change. Grunt work paid off. LLTextureFetch was also given a second options structure for texture fetches. Same as the original but with header return to caller requested. Baked textures should use this, the other 20,000 texture fetch requests should continue to use the original.
2013-06-19SH-4252 Add second policy class for large mesh asset downloadsMonty Brandenberg
Added second mesh class as well as an asset upload class. Refactored initialization to use less code and more data to cleanly get http started. Modified mesh to use the new http class for large requests (>2MB for now). Added additional timeout setting to llcorehttp to distinguish connection timeout from transport timeout and are now using transport timeout values for large asset downloads that may need more time.
2013-05-30Merge. Pull viewer-release to get the new version scheme changes.Monty Brandenberg
2013-05-07SH-4139 Convert http downloaders and responders to llcorehttp patternsMonty Brandenberg
Initial work completed on linux, moving over to windows to do debug and refinement. This includes 5/6 handlers based on existing responders and use of llcorehttp for the mesh header fetch.
2013-04-15SH-4106 Significantly upgrade the HttpHeaders interface for SSB.Monty Brandenberg
Header container moves from a vector of raw lines to a vector of string pairs representing name/value pairs in headers. For incoming headers, we normalize the name to lowercase and trim it. Values are only left-trimmed. Outgoing headers are left as-is. Simple find() method for the common case, forward and reverse iterators for those few who need to do it themselves. The HTTP status line (e.g. 'HTTP/1.1 200 Ok') is no longer treated as a header to be returned to caller. Unit tests, as usual, were a bear but they absolutely ensured outgoing HTTP header conformance after the change. Grunt work paid off. LLTextureFetch was also given a second options structure for texture fetches. Same as the original but with header return to caller requested. Baked textures should use this, the other 20,000 texture fetch requests should continue to use the original.
2013-03-29Update Mac and Windows breakpad builds to latestGraham Madarasz
2013-03-13Merging LLCURL::Responder changes with CHUI changes. Fixed gcc 4.6 compile ↵Don Kjer
failures
2013-03-13Large changes to the LLCurl::Responder API, as well as pulling in some ↵Don Kjer
changes to common libraries from the server codebase: * Additional error checking in http handlers. * Uniform log spam for http errors. * Switch to using constants for http heads and status codes. * Fixed bugs in incorrectly checking if parsing LLSD xml resulted in an error. * Reduced spam regarding LLSD parsing errors in the default completedRaw http handler. It should not longer be necessary to short-circuit completedRaw to avoid spam. * Ported over a few bug fixes from the server code. * Switch mode http status codes to use S32 instead of U32. * Ported LLSD::asStringRef from server code; avoids copying strings all over the place. * Ported server change to LLSD::asBinary; this always returns a reference now instead of copying the entire binary blob. * Ported server pretty notation format (and pretty binary format) to llsd serialization. * The new LLCurl::Responder API no longer has two error handlers to choose from. Overriding the following methods have been deprecated: ** error - use httpFailure ** errorWithContent - use httpFailure ** result - use httpSuccess ** completed - use httpCompleted ** completedHeader - no longer necessary; call getResponseHeaders() from a completion method to obtain these headers. * In order to 'catch' a completed http request, override one of these methods: ** httpSuccess - Called for any 2xx status code. ** httpFailure - Called for any non-2xx status code. ** httpComplete - Called for all status codes. Default implementation is to call either httpSuccess or httpFailure. * It is recommended to keep these methods protected/private in order to avoid triggering of these methods without using a 'push' method (see below). * Uniform error handling should followed whenever possible by calling a variant of this during httpFailure: ** llwarns << dumpResponse() << llendl; * Be sure to include LOG_CLASS(your_class_name) in your class in order for the log entry to give more context. * In order to 'push' a result into the responder, you should no longer call error, errorWithContent, result, or completed. * Nor should you directly call httpSuccess/Failure/Completed (unless passing a message up to a parent class). * Instead, you can set the internal content of a responder and trigger a corresponding method using the following methods: ** successResult - Sets results and calls httpSuccess ** failureResult - Sets results and calls httpFailure ** completedResult - Sets results and calls httpCompleted * To obtain information about a the response from a reponder method, use the following getters: ** getStatus - HTTP status code ** getReason - Reason string ** getContent - Content (Parsed body LLSD) ** getResponseHeaders - Response Headers (LLSD map) ** getHTTPMethod - HTTP method of the request ** getURL - URL of the request * It is still possible to override completeRaw if you want to manipulate data directly out of LLPumpIO. * See indra/llmessage/llcurl.h for more information.
2013-02-28Improve curl/ares init behavior by duping handles instead of calling easy_initGraham Madarasz (Graham)
2013-01-08SH-3468 WIP add memory tracking base classRichard Linden
more fixes for unit test crashes added llcommon initialization/teardown for unit tests that indirectly trigger lltrace changed access of atomic refcount to use preincrement/decrement operators to reflect desired semantics always call apr_initialize in LLCommon::initClass, even if already initialized...apr does internal reference counting to keep things straight
2012-12-21SH-3331 Fix incorrect overrun message when sending body with HTTPMonty Brandenberg
request. During readcallback, would generate an overrun-type message about reading position beyond end-of-data. Mistake was is messaging when state is exactly at end of data (which is expected) versus an overrun. Both result in declaring end-of-data to libcurl. Also changed some of the status logging for the metrics payload to be less chatty on success, more informative on error.
2012-12-10merge changes for DRTVWR-260Oz Linden
2012-12-07Removed duplicated block of code in LLAddBuildTest.cmake. Added comment to ↵Logan Dethrow
point to duplicated code. Replaced hard-coded tcmalloc link option with variable that is created in GooglePerfTools.cmake.
2012-12-04merge changes for BUG-768Nat Goodspeed