summaryrefslogtreecommitdiff
path: root/indra/newview/lltexturefetch.h
AgeCommit message (Collapse)Author
2012-09-05Merge in latest viewer-developmentsimon@Simon-PC.lindenlab.com
2012-08-21SH-3325 texture load slow on some machinesMonty Brandenberg
This doesn't really address 3325 directly but it is the result of research done while hunting it down. First, this is a thread safety improvement for canceled requests that have gone into resource wait state. I don't think we've seen a failure there but there was a window. It also cleans the resource wait queue earlier which lets us do less work and get requests more quickly into llcorehttp by bypassing the resource wait state. With this, I finally feel comfortable about rundown of requests.
2012-08-19SH-3329 Cached scene loads slower than same scene with cleared cache.Monty Brandenberg
A/B comparison with original code showed the newer issuing lower-priority requests of the cache reader and some other minor changes. Brought them into agreement (this is cargo-cult programming). Made the HTTP resource semaphore an atomic int for rigorous correctness across threads. I swear I'm going to tear down this code someday.
2012-08-17SH-3327 HTTP transfers not showing up in texture console 'net tot tex' value.Monty Brandenberg
Dropped an argument during integration which made the total byte count read lower than expected. Everything else is fine, however.
2012-08-07SH-3275 WIP Run viewer metrics for object update messagesRichard Linden
cleaned up LLStat and removed unnecessary includes
2012-07-19SH-3280 Better init/shutdown functionality for llcorehttp by llappviewerMonty Brandenberg
Isolate llcorehttp initialization into a utility class (LLAppCoreHttp) that provides glue between app and library (sets up policies, handles notifications). Introduce 'TextureFetchConcurrency' debug setting to provide some field control when absolutely necessary.
2012-07-02Merge 3.3.3 release with Drano HTTP library at 3.3.0Monty Brandenberg
Big delta was converting the new texture debugger support code to the new library. Viewer manifest should probably get an eyeball before release.
2012-06-26fix merge conflicts: v-r to dranoXiaohong Bao
2012-06-26v-r to drano merge conflicts wipBrad Payne (Vir Linden)
2012-06-22SH-3177, SH-3180 std::iostream and LLSD serialization for BufferArray objects.Monty Brandenberg
Seems to be working correctly. Not certain this is the fastest possible way to provide a std::streambuf interface but it's visually acceptable.
2012-06-18Tidy Texture Console, add cache & resource wait stats, issue stats line to ↵Monty Brandenberg
log on exit. With much trial-and-error, cleaned up the banner on the texture console and made everything mostly fit. Added global cache read, cache write and resource wait count events to the console display to show if cache is working. On clean exit, emit a log line to report stats to log file (intended for automated tests, maybe): LLTextureFetch::endThread: CacheReads: 2618, CacheWrites: 117, ResWaits: 0, TotalHTTPReq: 117
2012-06-14LLMutex recursive lock, global & per-request tracing, simple GET request, ↵Monty Brandenberg
LLProxy support, HttpOptions starting to work, HTTP resource waiting fixed. Non-LLThread-based threads need to do some registration or LLMutex locks taken out in these threads will not work as expected (SH-3154). We'll get a better solution later, this fixes some things for now. Tracing of operations now supported. Global and per-request (via HttpOptions) tracing levels of [0..3]. The 2 and 3 levels use libcurl's VERBOSE mode combined with CURLOPT_DEBUGFUNCTION to stream high levels of detail into the log. *Very* laggy but useful. Simple GET request supported (no Range: header). Really just a degenrate case of a ranged get but supplied an API anyway. Global option to use the LLProxy interface to setup CURL handles for either socks5 or http proxy usage. This isn't really the most encapsulated way to do this but a better solution will have to come later. The wantHeaders and tracing options are now supported in HttpOptions giving per-request controls. Big refactoring of the HTTP resource waiter in lltexturefetch. What I was doing before wasn't correct. Instead, I'm implementing the resource wait after the Semaphore model (though not using system semaphores). So instead of having a sequence like: SEND_HTTP_REQ -> WAIT_HTTP_RESOURCE -> SEND_HTTP_REQ, we now do WAIT_HTTP_RESOURCE -> WAIT_HTTP_RESOURCE2 (actual wait) -> SEND_HTTP_REQ. Works well but the prioritized filling of the corehttp library needs some performance work later.
2012-06-08Implemented HTTP retry for requests. Went in rather easily whichMonty Brandenberg
surprised me. Added a retry queue similar to ready queue to the policy object which is sorted by retry time. Currently do five retries (after the initial try) delayed by .25, .5, 1, 2 and 5 seconds. Removed the retry logic from the lltexturefetch module. Upped the waiting time in the unit test for the retries. People won't like this but tough, need tests.
2012-06-07Lock/Mutex fixes, documentation, new resource wait state, dtor cleanupsMonty Brandenberg
Went through all the code and tried to document lock and thread usage in the module. There's a huge comment block introducing all of this at the beginning and I believe it's correct (though not quite complete). Keep it updated, people. Added a new state, WAIT_HTTP_RESOURCE, that's sort of a side-state of SEND_HTTP_REQ. If we hit a high-water mark for HTTP requests, the extra are shunted to the new state once. Once levels fall to a low-water mark, we run through a wait list of UUIDs, sort the valid ones by priority and release them for service. This keeps the HTTP layer busy while leaving the active queue shallow enough that requests can still be re-prioritzed cheaply. Priority model changed. The new state uses the PRIORITY_LOW mask, the old users of _LOW are now at PRIORITY_NORMAL and sleepers woken up after an external event are kicked off at PRIORITY_HIGH. This combination along with the new state should avoid priority inversion and keep things running without resorting to an infinite pipeline. New state displays as "HTW" with green text in the texture console. Request cancelation and worker run-down should now be more correct but this edge case may need more attention.
2012-06-01Major steps towards implementing the policy component.Monty Brandenberg
Identified and reacted to the priority inversion problem we have in texturefetch. Includes the introduction of a priority_queue for the requests that are ready. Start some parameterization in anticipation of having policy_class everywhere. Removed _assert.h which isn't really needed in indra codebase. Implemented async setPriority request (which I hope I can get rid of eventually along with all priorities in this library). Converted to using unsigned int for priority rather than float. Implemented POST and did groundwork for PUT.
2012-05-29fo SH-3074: clear ghost and stale texture fetching requests in time.Xiaohong Bao
2012-05-24fix for VWR-28996: Texture stalls and threading modelsXiaohong Bao
2012-05-23Integrate llcorehttp library into lltexturefetch design.Monty Brandenberg
This is the first functional viewer pass with the HTTP work of the texture fetch code performed by the llcorehttp library. Not exactly a 'drop-in' replacement but a work-alike with some changes (e.g. handler notification in consumer thread versus responder notification in worker thread). This also includes some temporary changes in the priority scheme to prevent the kind of priority inversion found in VWR-28996. Scheme used here does provide liveness if not optimal responsiveness or order-of-operation. The llcorehttp library at this point is far from optimally performing. Its worker thread is making relatively poor use of cycles it gets and it doesn't idle or sleep intelligently yet. This early integration step helps shake out the interfaces, implementation niceties will be covered soon.
2012-05-10for SH-3132. roll back http concurrency for texture fetching back to 8.Xiaohong Bao
2012-05-10fix for SH-3132: Many "HTTP GET failed errors" with Drano merge candidate 255539Xiaohong Bao
2012-05-08For SH-3086. Make the fetch debugger not to stall the main thread.Xiaohong Bao
2012-05-04fix crashes and flaws for SH-3086Xiaohong Bao
2012-05-03For SH-3086: add functions to refetch all previous textures through HTTP and ↵Xiaohong Bao
from cache
2012-05-01for SH-3118: add a toggle "TextureFetchDebuggerEnabled" to turn the console ↵Xiaohong Bao
on and off.
2012-04-27fix some flaws for SH-3104Xiaohong Bao
2012-04-27Pull merge from drano-sh-3086Merov Linden
2012-04-26SH-3104 : Implemented HTTP bulk fetch for texture debug consoleMerov Linden
2012-04-26For SH-3121: collect time of reloading all textures from cache and through ↵Xiaohong Bao
HTTP when all objects loading are done.
2012-04-25for subtasks SH-3118, SH-3112, SH-3110, SH-3106, SH-3091 for SH-3086:Xiaohong Bao
As a viewer architect, I would like to understand how fast each of the components of the texture pipeline can run in isolation
2012-03-15added stats tracking for texture/object cache hit rates and texture cache ↵Richard Linden
read latency
2011-12-05SH-2652 WIP -- Add timers to relevant areas, pause render pipeline while ↵Dave Parks
occlusion queries from previous frame are still pending and perform texture decode work.
2011-01-21Merge viewer-development-shining up to REV 3bf724ecb7c6 (fix for SH-829: ↵Xiaohong Bao
Viewer attempting to load precached images in file types that are not being used.)
2010-12-16ESC-237 No static init of LLAtomics and move TFRequest out of unnamed namespace.Monty Brandenberg
Linux startup crash appears to be due to static/global C++ init of LLAtomic types. The initializer with explicit value makes some runtime calls and it looks like these assume, at least on Linux, that apr_initialize() has been called. So move the static POST count to a member and provide accessors and increment/decrement. Command queue was built on a pointer to a class in anonymous namespace and that's not quite valid. Made it a nested class (really a nested forward declaration) while keeping the derived classes in anonymous.
2010-12-13debug code for SH-639: http requests 3X more in viewer 2.4Xiaohong Bao
2010-12-10Decided to refactor a bit. Was using LLSD as an internal dataMonty Brandenberg
representation transferring ownership, doing data aggregation in a very pedantic way. That's just adding unneeded cost and complication. Used the same objects to transport data as are collecting it and everything got simpler, faster, easier to read with fewer gotchas. Bit myself *again* doing the min/max/mean merges but the unittests where there to pick me up again. Added a per-region FPS metric while I was at it. This is much asked for and there was a convenient place to sample the value.
2010-12-02ESC-211 Metrics data sink - fix delivery by viewerMonty Brandenberg
The TextureFetch thread was still stalling out due to a different path that determines whether there is work or not in the thread (uses getPending()) and that had to be harmonized with the changes to runCondition(). I'm not happy with this solution but a refactor of the LLThread tree isn't in the cards right now.
2010-11-29ESC-154 ESC-156 Now using region hash rather than region uuid as identifier.Monty Brandenberg
In the viewer, the region's UUID is acquired very late and isn't generally used as the canonical region identifier. The U64 region hash is a better and more consistently used identifier so I'm moving over to using that as the region key. Don't have a proper reserved invalid region hash which is unfortunate, but then, so much is.
2010-11-24ESC-154 ESC-156 Metrics integration across threadsMonty Brandenberg
Using unpause() method in derived class rather than wake() in furthest base class solved the stalling problem. I still think too many levels of the LLTextureFetch hierarchy are keeping thread state, however. The LLViewerRegion instance an agent enters doesn't necessarily have its region_id yet, that only comes after the handshake, if any. So add a few more metrics insertion points to propagate region into metrics. Finally, try to launch a final metrics report when a quit is initiated.
2010-11-23ESC-154 ESC-156 Integrating metrics collector into viewer.Monty Brandenberg
After discussions, renamed 'QAModeMetricsSubmetrics' to 'QAModeMetrics' and confirmed that LLCachedControl<> is the way to go. Moved the resulting flag out of LLViewerAssetStats (where it didn't belong) and it lives in both LLAppViewer and LLTextureFetch where it does belong.
2010-11-22ESC-154 ESC-156 Metrics integration into viewer's threadsMonty Brandenberg
Removed declared but undefined interfaces from LLTextureFetch family. Had inserted the cross-thread command processor into some of LLTextureFetchWorker's processing which was unnatural and probably wrong. Moved it to LLTextureFetch which turned out to be far, far more natural. Better documentation on the asLLSD() format. Refined LLSD stats merger logic and enhanced unit tests to verify same.
2010-11-18ESC-154 ESC-155 ESC-156Monty Brandenberg
Asset fetch requests wrapped to allow for measurements. Asset fetch enqueues, dequeues and completion times recorded to asset stats collector. Texture fetch operations (http and udp) recorded to asset stats collector. Stats collector time vallue switched from F32 to U64 which is the more common type in the viewer. Cross-thread mechanism introduced to communicate region changes and generate global statistics messages. Facility to deliver metrics via Capabilities sketched in but needs additional work. Documentation and diagrams added.
2010-08-24Backed out changeset c3d41f18ce2bTofu Linden
back-out the back-out for this branch. yay.
2010-08-24Backed out changeset a62bf7c0af21Tofu Linden
Backing out this merge that I pushed (prematurely) to the wrong place.
2010-08-24merge heads. whew.Tofu Linden
2010-08-13Change license from GPL to LGPL (version 2.1)Oz Linden
2010-08-11fix for EXT-7399: 2.0.2 with http textures loads textures slower than 1.23.5 andXiaohong Bao
EXT-7397: decoding textures get stuck in REQ status
2010-08-04more fix for EXT-7397: decoding textures get stuck in REQ statusXiaohong Bao
2010-08-03fix for EXT-7399: 2.0.2 with http textures loads textures slower than 1.23.5Xiaohong Bao
and EXT-7397: decoding textures get stuck in REQ status
2010-05-04EXT-7080 Partial FIXED: Outfit texture updates are not transmitted (in ↵Xiaohong Bao
either direction) when HTTP Textures is enabled Fixed baked textures downloading. Uploading will be fixed on the server side.
2010-03-04fix for EXT-5683: viewer crashes at llcommon/llworkerthread.cpp(323): ERROR: ↵Xiaohong Bao
LLWorkerClass::checkWork: ASSERT(workreq).