summaryrefslogtreecommitdiff
path: root/indra/llcorehttp
AgeCommit message (Collapse)Author
2015-01-13merge changes for 3.7.24-releaseOz Linden
2014-11-14Remove a couple tests for unsigned long < 0.Nat Goodspeed
clang correctly notes that such tests add nothing to the runtime behavior of a program. Unfortunately, clang notes that in the form of a compile error.
2014-11-14Automated merge with ssh://bitbucket.org/lindenlab/viewer-releaseNat Goodspeed
2014-11-06BUG-7698, BUG-7688, BUG-7694 (others) CDN connection issues.Monty Brandenberg
Under pipelining, requests were given a 5x timeout factor due to the way that the timeout clock works in libcurl. Under CDN load, connections were not being torn down quickly and it was only this timer that led to disconnect and retry. So, we want to break a connection that isn't making progress but that isn't immediately possible. We'll compromise with a 60S timeout that (we hope) will be neither too long for stalled connections nor too short for large asset transfer requests.
2014-10-17Update to build on Xcode 6.0: remove unused codecallum_linden
2014-10-17Update to build on Xcode 6.0: clang detecting unsigned int (size_t) ↵callum_linden
comparison with <0 [-Wtautological-compare]
2014-10-17Update to build on Xcode 6.0: interesting - clang doesn't like it if you ↵callum_linden
specify extra components of a format string that aren't populated in snprintf
2014-10-10MAINT-4564 HTTP Pipelining is not happening in Drano HTTP Phase 4Monty Brandenberg
Incorporate the new libcurl 7.38.0 build with curl bug 1420 workaround. Add developer-centric testing code to evaluate the workaround or a future fix for 1420.
2014-09-22Tuning and documentation. Use a fast poll frequency (0.05S)Monty Brandenberg
on the HTTP requests for inventory. We'll benchmark with that and see how it goes. Document some of the history of the background fetcher for future devs. Suggest some future projects to make things faster. Pointers on using LLSD with the llcorehttp library in the readme. And restructured the LLSD onCompleted() processing phases using do{}while(false) which produced a code flow that is fairly attractive.
2014-09-19Introduce libcurl handle cache. Create a private cacheMonty Brandenberg
of used handles and a fast handle factory that's thread- correct.
2014-09-18Documentation. Describe curl bug 1420 testing and how toMonty Brandenberg
reproduce data corruption via timeouts.
2014-09-04Pipelining work. Extend transfer timeout by the pipeline depthMonty Brandenberg
as transfers can appear delayed with deep pipelining and more requests in the pool. Added bad HTTP status error (typically getting a 0 back as HTTP status from libcurl) to the list of retryable errors. There's a response stream problem with libcurl and pipelining that induces this problem. Retrying helps but may not be entirely safe. Watch bug 1420 on the libcurl sourceforge bug tracker. Extend options of test/example program to include un-ranged requests. Document the excessive data transfer induced when ranged requests are disabled. This is an abnormal mode for very rare users so we'll just eat that for now.
2014-08-12Better support for dynamic option changes in llcorehttp. Libcurl hasMonty Brandenberg
some problems disabling pipelining on a multi handle with outstanding requests so build a more conservative system that allows requests to drain before setting curl multi options. Would rather not have this but it is significantly safer. "HttpPipelining" debug setting is now fully dynamic. Connection limits can also be made dynamic in the near future. Upped the default connection count back to 8 for now but will revisit this in the tuning phase. It might be time to combine mesh and textures into a single asset class. For normal server operations that would be a clear path, but for server under load, the current scheme may be better. Minor cleanup in logging to elminate some redundant strings. Might add some more tracing to the stall logic 'just in case'.
2014-07-03Add pipelining and tracing command line options to the test program.Monty Brandenberg
2014-06-23First HTTP pipelining viewer. Enable pipelining forMonty Brandenberg
GetTexture and GetMesh2 at a pipeline depth of 5. Create global debug option, HttpPipelining, to enable and disable HTTP pipelining (defaults to true). Tweak texture and mesh low- and high-water request levels based on pipelining status and depth. Fixup texture console which was damaged in a recent release. Split logging of the no-request HTTP error case into two cases: one for missing URL in HTTP request, one for HTTP request not created. A refactor in llcorehttp is coming: I will be moving all libcurl- using code into libcurl-specific modules.
2014-05-13sunshine-external merge WIPBrad Payne (Vir Linden)
2014-02-25merge viewer-release to sunshine-externalBrad Payne (Vir Linden)
2014-02-24merge with releaseRichard Linden
2014-01-09fixed unit tests being generated as Windows GUI projects instead of console ↵Richard Linden
projects
2013-10-16mergeBrad Payne (Vir Linden)
2013-10-08merge from viewer-releaseRichard Linden
2013-10-04Convert one more unit test over to improved waiting scheme to avoid build ↵Monty Brandenberg
failures.
2013-10-04Merge. Pull in viewer-release after 3.6.7 release.Monty Brandenberg
2013-09-27Up the transfer timeout of small meshes to 120S. This matchesMonty Brandenberg
the change made for MAINT-2347. Large transfers are still 10 minutes. Add/update to-do list and add some more info to the FAQ in the Readme.
2013-09-24SH-3690 SH-4505 Cleanup pass through code.Monty Brandenberg
Start using DNS cache in legacy LLCurl code. Go to 15 seconds particularly as we're using threaded resolver at this point. Documentation cleanup. Add libcurl status checking and logging for curl_easy_setopt() operations that fail. Shouldn't happen and we'll just continue anyway but there's info in the logs to track these down now. Cleaned up logic around FASTTIMER enable defines used to evaluate pipeline stalls in main thread. Removed long-standing thread race around caps strings and URL construction. Not a significant risk but refactoring the code to get rid of them removed one huge eyesore. It can be made even slicker if desired (see notes).
2013-09-18SH-4492 Create a useful README for llcorehttpMonty Brandenberg
Last bit for this release. Describe stream adapters and how to select a policy class. Slight changes to setup code to make reality reflect documentation.
2013-09-17SH-4492 Create a useful README for llcorehttp.Monty Brandenberg
First edit complete. Use the library in 15 minutes. Describe the code. Refinements to the initial try. Describe that code. Still to do: more refinements, how to choose a policy class, FAQ.
2013-09-11SH-4490 More 'humane' error code presentation from llcorehttp callersMonty Brandenberg
Added toTerseString() conversion on HttpStatus to generate a string that's more descriptive than the hex value of the HttpStatus value but still forms a short, searchable token (e.g. "Http_503" or "Core_7"). Using this throughout the viewer now, no live cases of toHex(), I believe.
2013-09-11SH-4489 New debug/dev settings for control over new mesh behaviorMonty Brandenberg
Added 'MeshUseGetMesh1' and 'MeshUseHttpRetryAfter' debug settings to control mesh transport behavior. First forces the use of the legacy mesh fetch style with high concurrency and connection churn. The second, on by default, honors Retry-After values if they are reasonable. If off or unreasonable, internal delay times are used.
2013-08-19Add 'internal'/'external' token to DEBUG retry message so that dev/QAMonty Brandenberg
can know exactly where a retry value was sourced.
2013-08-16SH-4407 Tuning to get new code working as well.Monty Brandenberg
Do some runtime code avoidance and skip unnecessary libcurl and syscall invocations.
2013-07-30SH-4371 Reduce 22mS inter-connection latency.Monty Brandenberg
This really extended into the client-side request throttling. Moved this from llmeshrepository (which doesn't really want to do connection management) into llcorehttp. It's now a class option with configurable rate. This still isn't the right thing to do as it creates coupling between viewer and services. When we get to pipelining, this notion becomes invalid.
2013-07-29Reduce HTTP request retry log spam. Thought I'd done this in anMonty Brandenberg
earlier maintenance branch but the code never showed up. I'll do it again. Spam is still available by bumping 'CoreHttp' tag up to DEBUGS level logging. Needed for QA. Can also get this data from tracing.
2013-07-29SH-4371 Reduce 22mS inter-conenction latency on HTTP operations.Monty Brandenberg
Simple change dropped this value by 7-10mS or so. Any time we complete an operation on a transport pass, arrange to skip the run-loop sleep so that we fill a possible empty slot as quickly as possible. With pipelining, this kind of thing should become unnecessary but for now, we'll do this to increase throughput.
2013-07-29SH-4368 Adjust upload timeout parameters for slow networks.Monty Brandenberg
Generally sorted the mesh timeout parameters for maximum transport time (staying with default 30 for connect). 60S for normal meshes, 600S for large. Also documented default option values in httpoptions.h. Useful to have these. In the future, the timeouts might go into standard llsd options where they can be tracked a bit more.
2013-07-12SH-4312 Configuration data between viewer and llcorehttp is clumsy.Monty Brandenberg
Much improved. Unified the global and class options into a single option list. Implemented static and dynamic setting paths as much as possible. Dynamic path does require packet/RPC but otherwise there's near unification. Dynamic modes can't get values back yet due to the response/notifier scheme but this doesn't bother me. Flatten global and class options into simpler struct-like entities. Setter/getter available on these when needed (external APIs) but code can otherwise fiddle directly when it knows what to do. Much duplicated options/state removed from HttpPolicy. Comments cleaned up. Threads better described and consistently mentioned in API docs. Integration test extended for 503 responses with Reply-After headers.
2013-07-08Found the memory corruptor. String trimmer didn't have a validMonty Brandenberg
termination test. Sheesh. Also get some more numbers out of the example/load test program which drives traffic. This should give some useful insights into how the current http throttle works (or doesn't) with varying client demands.
2013-07-03Update the example program so it handles meshes as well. Fix theMonty Brandenberg
request feed logic to use high/low-water level logic as is done in viewer code.
2013-06-27SH-4310/BUG-2810/MAINT-2794 Better status checking and error logging in ↵Monty Brandenberg
Mesh code. Pay correct attention to status codes coming back from services. Generate better and consistent error messages when problems arise. There's more to do in error handling, need a way to cleanly fail all request types, only have that for LOD at this point. Do better keeping the HTTP pipeline between the low and high water marks. This was made challenging because the outer most code couldn't really see what's going on internally (whose actions are delayed in a worker thread). More to do here, the debug-like requests don't honor limits, that will come later. Made retry counts available from llcorehttp which can be used by the throttle-anticipating logic to advance the count. It helps but it reinforces the coupling between viewer and server which I do not like.
2013-06-20SH-4257 Preparation for a new cap grant: GetMesh2Monty Brandenberg
Mesh repo is using three policy classes now: one for large objects, one for GetMesh2 regions, one for GetMesh regions. It's also detecting the presence of the cap and using the correct class. Class initialization cleaned up significantly in llappcorehttp using data-directed code. Pulled in the changes to HttpHeader done for sunshine-internal then did a refactoring pass on the header callback which now uses a unified approach to clean up and deliver header information to all interested parties. Added support for using Retry-After header information on 503 retries.
2013-06-05merge with viewer-releaseRichard Linden
2013-05-21mergeBrad Payne (Vir Linden)
2013-05-14SH-4190 Reactivate unit tests neutered under MAINT-3202Monty Brandenberg
HttpStatus unit tests have never caused a unit test failure but they do have the word 'error' in their text descriptions which gets picked up by the log processor in TC builds. So, reactivate the tests and reform the descriptions.
2013-05-14MAINT-2302 Tests frequently fail on Windows builds.Monty Brandenberg
I'm calling this the more correct fix. The httprequest tests actually talk to an HTTP server running in the Python test scaffold script. Under severe CPU competition, it may not get the cycles needed to start up and make progress. So this modifies the test to spin a little faster and dwell waiting on the server for a longer period. Hope this will be adequate to make the tests reliable.
2013-05-13Merge. Refresh from /lindenlab/viewer-cat prior to pushing.Monty Brandenberg
2013-05-13Whinge reduction effort on the unit tests. Python 2.7 SocketServerMonty Brandenberg
library has new exception-throwing behavior when a client disconnects unannounced. Generally ignore exceptions as a result as we don't care about the server side. On HTTP trace-mode tests, spin a little faster and longer to give libcurl time to emit all the junk it wants to send us. Should reduce 'reasonable time' failures on tests <12> and <13>.
2013-05-09Merge downstream code and viewer-betasimon
2013-05-06BUG-2295/MAINT-2624 unexpected crash around Content-Range: header processingMonty Brandenberg
Not certain what the source of the short data is with one resident but I'm going to make these problems retryable as they are transport-related. Lift the retry detection into a method that should be reusable by others interested in determining what is retryable. Trace output handling on the libcurl debug callback was attrocious. Some unsafe length handling on my part was protected by a second layer of defense. Made that correct and more useful by logging actual data sizes during trace.
2013-04-29BUG-2295/MAINT-2624 [FIXED] Crash in HttpOpRequest::stageFromActive w/ ↵Monty Brandenberg
Content-Range Don't rely on a response body being present should a Content-Range header be parsed. Unit tests captured the original crash and confirm the fix.
2013-04-25Revert ares and libcurl version update that was causing problems, revise curlsimon
handle duplication code. Reviewed by Kelly