| Age | Commit message (Collapse) | Author |
|---|
|
numbers.
This is a command-line utility to pull content down from a service through
the llcorehttp library to produce timings and resource footprints.
|
|
LLProxy support, HttpOptions starting to work, HTTP resource waiting fixed.
Non-LLThread-based threads need to do some registration or LLMutex locks taken out in these
threads will not work as expected (SH-3154). We'll get a better solution later, this fixes
some things for now. Tracing of operations now supported. Global and per-request (via
HttpOptions) tracing levels of [0..3]. The 2 and 3 levels use libcurl's VERBOSE mode
combined with CURLOPT_DEBUGFUNCTION to stream high levels of detail into the log. *Very*
laggy but useful. Simple GET request supported (no Range: header). Really just a
degenrate case of a ranged get but supplied an API anyway. Global option to use the
LLProxy interface to setup CURL handles for either socks5 or http proxy usage. This
isn't really the most encapsulated way to do this but a better solution will have to
come later. The wantHeaders and tracing options are now supported in HttpOptions giving
per-request controls. Big refactoring of the HTTP resource waiter in lltexturefetch.
What I was doing before wasn't correct. Instead, I'm implementing the resource wait
after the Semaphore model (though not using system semaphores). So instead of having
a sequence like: SEND_HTTP_REQ -> WAIT_HTTP_RESOURCE -> SEND_HTTP_REQ, we now
do WAIT_HTTP_RESOURCE -> WAIT_HTTP_RESOURCE2 (actual wait) -> SEND_HTTP_REQ. Works
well but the prioritized filling of the corehttp library needs some performance
work later.
|
|
The NORMAL range doesn't do any sleeping at all and so we'll
spin the core harder than we already are. Bring all idlers
into the same range.
|
|
For now, workaround...
|
|
Implemented/modified PUT & POST to not used chunked encoding for the request.
Made the unit test much happier and probably a better thing for the pipeline.
Have a cheesy static & dynamic proxy capability using both local options and
a way to wire into LLProxy in llmessages. Not a clean thing but it will get
the proxy path working with both socks5 & http proxies. Refactoring to get
rid of unneeded library handler and unified an HttpStatus return for all
requests. Big batch of code removed as a result of that and more is possible
as well as some syscall avoidance with a bit more work. Boosted the unit
tests for simple PUT & POST test which revealed the test harness does *not*
like chunked encoding so we'll avoid it for now (and don't really need it
in any of our schemes).
|
|
Beware of bad documentation. operator--(int) does not return what
the header claimed it did.
|
|
This brings in a copy of llmessage's llsdmessage testing server. We run
a mocked HTTP service to handle requests and the integration tests run
against it by picking up the LL_TEST_PORT environment variable when running.
Add some checks and output to produce useful info when run in the wrong
environment and when bad status is received. Later will add a dead port
as well so we can test that rather than use 'localhost:2'.
|
|
HttpRequest::update() honor time limit.
Generally, opaque data operations are expected to be over 'void *' and have
now converted interfaces to do that. Update() method honors millisecond limit to dwell
time. Might want to homologate the millis/uSecs mix later....
|
|
206/content-range hack in xport.
Retry/response handling is decided in policy so moved that there. Removed special case
206-without-content-range response in transport. Have this sitation recognizable in the
API and let callers deal with it as needed.
|
|
cleanup.
Our logging holds on to a changing bit of memory between operations and the memory
leak detection I'm using senses this and complains. So, for now, disable the
final memory check on Mac & Linux, leave it active on Windows. Solve this for
real some other day. Add try/catch blocks to do cleanup in unit tests that go
wrong so that we don't get a cascade of assertion failures when subsequent tests
find singletons still alive.
|
|
|
|
|
|
surprised me. Added a retry queue similar to ready queue to the
policy object which is sorted by retry time. Currently do five
retries (after the initial try) delayed by .25, .5, 1, 2 and 5
seconds. Removed the retry logic from the lltexturefetch module.
Upped the waiting time in the unit test for the retries. People
won't like this but tough, need tests.
|
|
|
|
|
|
|
|
|
|
who want to try IPV6 can still override at will using CURLOPT_IPRESOLVE.
|
|
|
|
|
|
|
|
now avoiding doing HTTP fetches for read data. Not certain it's
completely correct but the difference is already significant.
|
|
Went through all the code and tried to document lock and thread usage
in the module. There's a huge comment block introducing all of this
at the beginning and I believe it's correct (though not quite complete).
Keep it updated, people. Added a new state, WAIT_HTTP_RESOURCE, that's
sort of a side-state of SEND_HTTP_REQ. If we hit a high-water mark
for HTTP requests, the extra are shunted to the new state once. Once
levels fall to a low-water mark, we run through a wait list of UUIDs,
sort the valid ones by priority and release them for service. This
keeps the HTTP layer busy while leaving the active queue shallow enough
that requests can still be re-prioritzed cheaply. Priority model
changed. The new state uses the PRIORITY_LOW mask, the old users
of _LOW are now at PRIORITY_NORMAL and sleepers woken up after an
external event are kicked off at PRIORITY_HIGH. This combination
along with the new state should avoid priority inversion and keep
things running without resorting to an infinite pipeline. New
state displays as "HTW" with green text in the texture console.
Request cancelation and worker run-down should now be more
correct but this edge case may need more attention.
|
|
|
|
|
|
|
|
Over the years we've skipped more and more of the tests in
llhttpclient_test.cpp (nee llhttpclient_tut.cpp) because they've relied on
particular behaviors from Internet sites not under our control.
We skipped a test that fetches llsd+xml from secondlife.com because
"secondlife.com is not reliable enough for unit tests."
We skipped a test that tries to observe a failure with "http://www.invalid"
because some local ISPs turn "no such domain" DNS errors into valid pages
offering the requester to buy the specified domain name.
Today we've had to skip tests attempting to contact "http://www.google.com"
for reasons we haven't yet diagnosed, but that probably have to do with
Google's IPv6 rollout.
Use local temp server test_llsdmessage_peer.py as the success destination,
eliminating DNS, Internet access and remote server behavior as failure modes.
Use idle localhost port for failure test.
Re-enable all skipped LLHTTPClient tests!
Re-enable on Windows!
In support of these tests, modify test_llsdmessage_peer.py:
Support HEAD as no-data variant of GET.
Change GET result dict to avoid resembling an error response -- confusing.
Make GET/POST return actual dict rather than undecorated string "success".
Because of that last, change llcapabilitylistener_test.cpp and
llsdmessage_test.cpp to extract "reply" key from response rather than
expecting response to be a string itself.
|
|
indra/test/llhttpclient_tut.cpp ==> indra/llmessage/tests/llhttpclient_test.cpp,
with corresponding CMakeLists.txt changes in their respective directories.
This first commit merely moves the test source file without changing it --
separating out moves from edits simplifies code review.
|
|
Today, 2012-06-06, LLHTTPClient::get("http://www.google.com/") has stopped
working. This is unrelated to viewer code changes: it crosses repo boundaries.
Skip the afflicted tests to get builds working until we can recast the tests
to avoid the flaky dependency.
|
|
The change from LLProcessLauncher to LLProcess introduces the possibility of a
NULL (default-constructed) LLProcessPtr. Add certain static LLProcess methods
accepting LLProcessPtr, forwarding to nonstatic method when non-NULL but doing
something reasonable with NULL. Use these methods in LLPLuginProcessParent.
|
|
|
|
Implemented first global policy definitions to support SSL CA certificate configuration
to support https: operations. Fixed HTTP 206 status handling to match what is currently
being done by grid services and to lay a foundation for fixes that will be a response
to ER-1824. More libcurl CURLOPT options set on easy handles to do peer verification
in the traditional way. HTTP POST working and now reporting asset metrics back to
grid for the viewer's asset system. This uses LLSD so that is also showing as compatible
with the new library.
|
|
1 byte of data.
Shouldn't be making that kind of mistake.
|
|
fixing priorities.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
chunking data. Remove the stateful use of a seek pointer so
that shared read is possible (though maybe not interesting).
|
|
what normal requests do...
|
|
|
|
|
|
excuse to go through an use a typedef for priority and policy class id.
|
|
|
|
Identified and reacted to the priority inversion problem we
have in texturefetch. Includes the introduction of a priority_queue
for the requests that are ready. Start some parameterization in
anticipation of having policy_class everywhere. Removed _assert.h
which isn't really needed in indra codebase. Implemented async
setPriority request (which I hope I can get rid of eventually along
with all priorities in this library). Converted to using unsigned
int for priority rather than float. Implemented POST and did
groundwork for PUT.
|
|
|
|
|
|
|
