| Age | Commit message (Collapse) | Author |
|---|
|
Turns out that one of our WorkQueue integration tests was relying on the
incorrect runFor() behavior that we just fixed, so the test broke. Now that
runFor() doesn't wait around for work to be posted, use an explicit wait loop
instead.
To support this, add LLCond::get(functor), where functor must accept a const
reference to the stored data. This new get() returns whatever the functor
returns, allowing a caller to peek at the stored data.
Also use universal references for all remaining LLCond functor arguments.
|
|
Reverting a merge is sticky: it tells git you never want to see that branch
again. Merging the DRTVWR-546 branch, which contained the revert, into the
glthread branch undid much of the development work on that branch. To restore
it we must revert the revert.
This reverts commit 029b41c0419e975bbb28454538b46dc69ce5d2ba.
|
|
This reverts commit 5188a26a8521251dda07ac0140bb129f28417e49, reversing
changes made to 819088563e13f1d75e048311fbaf0df4a79b7e19.
|
|
|
|
|
|
Add a test exercising this feature.
|
|
DRTVWR-546
|
|
Also make workqueue_test.cpp more robust.
|
|
A typical WorkQueue has a string name, which can be used to find it to post
work to it. "Work" is a nullary callable.
WorkQueue is a multi-producer, multi-consumer thread-safe queue: multiple
threads can service the WorkQueue, multiple threads can post work to it.
Work can be scheduled in the future by submitting with a timestamp. In
addition, a given work item can be scheduled to run on a recurring basis.
A requesting thread servicing a WorkQueue of its own, such as the viewer's
main thread, can submit work to another WorkQueue along with a callback to be
passed the result (of arbitrary type) of the first work item. The callback is
posted to the originating WorkQueue, permitting safe data exchange between
participating threads.
Methods are provided for different kinds of servicing threads. runUntilClose()
is useful for a simple worker thread. runFor(duration) devotes no more than a
specified time slice to that WorkQueue, e.g. for use by the main thread.
|
|
|
|
ThreadSafeSchedule::tryPopUntil() (and therefore tryPopFor()) was simply
delegating to LLThreadSafeQueue::tryPopUntil(), with an adjusted timeout since
we want to wake up as soon as the head item, if any, becomes ready. But then
we have to loop back to retry the pop to actually deal with that head item.
In addition, ThreadSafeSchedule::popWithTime() was spinning rather than
properly blocking on a timed condition variable. Fixed.
|
|
ThreadSafeSchedule orders its items by timestamp, which can be passed either
implicitly or explicitly. The timestamp specifies earliest delivery time: an
item cannot be popped until that time.
Add initial tests.
Tweak the LLThreadSafeQueue base class to support ThreadSafeSchedule:
introduce virtual canPop() method to report whether the current head item is
available to pop. The base class unconditionally says yes, ThreadSafeSchedule
says it depends on whether its timestamp is still in the future.
This replaces the protected pop_() overload accepting a predicate. Rather than
explicitly passing a predicate through a couple levels of function call, use
canPop() at the level it matters. Runtime behavior that varies depending on
an object's leaf class is what virtual functions were invented for.
Give pop_() a three-state enum return so pop() can distinguish between "closed
and empty" (throws exception) versus "closed, not yet drained because we're
not yet ready to pop the head item" (waits).
Also break out protected tryPopUntil_() method, the body logic of
tryPopUntil(). The public method locks the data structure, the protected
method requires that its caller has already done so.
Add chrono.h with a more full-featured LL::time_point_cast() function than the
one found in <chrono>, which only converts between time_point durations, not
between time_points based on different clocks.
|
|
These functions allow prepending or removing an item at the left end of an
arbitrary tuple -- for instance, to add a sequence key to a caller's data,
then remove it again when delivering the original tuple.
|
|
|
|
Introduce Oz's LLERROR_CRASH macro analogous to the old LLError::crashAndLoop()
function. Change LL_ENDL macro so that, after calling flush(), if the CallSite
is for LEVEL_ERROR, we invoke LLERROR_CRASH right there.
Change the meaning of LLError::FatalFunction. It used to be responsible for
the actual crash (hence crashAndLoop()). Now, instead, its role is to disrupt
control flow in some other way if you DON'T want to crash: throw an exception,
or call exit() or some such. Any FatalFunction that returns normally will fall
into the new crash in LL_ENDL.
Accordingly, the new default FatalFunction is a no-op lambda. This eliminates
the need to test for empty (not set) FatalFunction in Log::flush().
Remove LLError::crashAndLoop() because the official LL_ERRS crash is now in
LL_ENDL.
One of the two common use cases for setFatalFunction() used to be to intercept
control in the last moments before crashing -- not to crash or to avoid
crashing, but to capture the LL_ERRS message in some way. Especially when
that's temporary, though (e.g. LLLeap), saving and restoring the previous
FatalFunction only works when the lifespans of the relevant objects are
strictly LIFO.
Either way, that's a misuse of FatalFunction. Fortunately the Recorder
mechanism exactly addresses that case. Introduce a GenericRecorder template
subclass, with LLError::addGenericRecorder(callable) that accepts a callable
with suitable (level, message) signature, instantiates a GenericRecorder, adds
it to the logging machinery and returns the RecorderPtr for possible later use
with removeRecorder().
Change llappviewer.cpp's errorCallback() to an addGenericRecorder() callable.
Its role was simply to update gDebugInfo["FatalMessage"] with the LL_ERRS
message, then call writeDebugInfo(), before calling crashAndLoop() to finish
crashing. Remove the crashAndLoop() call, retaining the gDebugInfo logic. Pass
errorCallback() to LLError::addGenericRecorder() instead of setFatalFunction().
Oddly, errorCallback()'s crashAndLoop() call was conditional on a compile-time
SHADER_CRASH_NONFATAL symbol. The new mechanism provides no way to support
SHADER_CRASH_NONFATAL -- it is a Bad Idea to return normally from any LL_ERRS
invocation!
Rename LLLeapImpl::fatalFunction() to onError(). Instead of passing it to
LLError::setFatalFunction(), pass it to addGenericRecorder(). Capture the
returned RecorderPtr in mRecorder, replacing mPrevFatalFunction. Then
~LLLeapImpl() calls removeRecorder(mRecorder) instead of restoring
mPrevFatalFunction (which, as noted above, was order-sensitive).
Of course, every enabled Recorder is called with every log message. onError()
and errorCallback() must specifically test for calls with LEVEL_ERROR.
LLSingletonBase::logerrs() used to call LLError::getFatalFunction(), check the
return and call it if non-empty, else call LLError::crashAndLoop(). Replace
all that with LLERROR_CRASH.
Remove from llappviewer.cpp the watchdog_llerrs_callback() and
watchdog_killer_callback() functions. watchdog_killer_callback(), passed to
Watchdog::init(), used to setFatalFunction(watchdog_llerrs_callback) and then
invoke LL_ERRS() -- which seems a bit roundabout. watchdog_llerrs_callback(),
in turn, replicated much of the logic in the primary errorCallback() function
before replicating the crash from llwatchdog.cpp's default_killer_callback().
Instead, pass LLWatchdog::init() a lambda that invokes the LL_ERRS() message
formerly found in watchdog_killer_callback(). It no longer needs to override
FatalFunction with watchdog_llerrs_callback() because errorCallback() will
still be called as a Recorder, obviating watchdog_llerrs_callback()'s first
half; and LL_ENDL will handle the crash, obviating the second half.
Remove from llappviewer.cpp the static fast_exit() function, which was simply
an alias for _exit() acceptable to boost::bind(). Use a lambda directly
calling _exit() instead of using boost::bind() at all.
In the CaptureLog class in llcommon/tests/wrapllerrs.h, instead of statically
referencing the wouldHaveCrashed() function from test.cpp, simply save and
restore the current FatalFunction across the LLError::saveAndResetSettings()
call.
llerror_test.cpp calls setFatalFunction(fatalCall), where fatalCall() was a
function that simply set a fatalWasCalled bool rather than actually crashing
in any way. Of course, that implementation would now lead to crashing the test
program. Make fatalCall() throw a new FatalWasCalled exception. Introduce a
CATCH(LL_ERRS("tag"), "message") macro that expands to:
LL_ERRS("tag") << "message" << LL_ENDL;
within a try/catch block that catches FatalWasCalled and sets the same bool.
Change all existing LL_ERRS() in llerror_test.cpp to corresponding CATCH()
calls. In fact there's also an LL_DEBUGS(bad tag) invocation that exercises an
LL_ERRS internal to llerror.cpp; wrap that too.
|
|
|
|
LLSDNotationFormatter (also LLSDNotationStreamer that uses it, plus
operator<<(std::ostream&, const LLSD&) that uses LLSDNotationStreamer) is most
useful for displaying LLSD to a human, e.g. for logging. Having the default
dump raw binary bytes into the log file is not only suboptimal, it can
truncate the output if one of those bytes is '\0'. (This is a problem with the
logging subsystem, but that's a story for another day.)
Use OPTIONS_PRETTY_BINARY wherever there is a default LLSDFormatter
::EFormatterOptions argument.
Also, allow setting LLSDFormatter subclass boolalpha(), realFormat() and
format(options) using optional constructor arguments. Naturally, each subclass
that supports this must accept and forward these constructor arguments to its
LLSDFormatter base class constructor.
Fix a couple bugs in LLSDNotationFormatter::format_impl() for an LLSD::Binary
value with OPTIONS_PRETTY_BINARY:
- The code unconditionally emitted a b(len) type prefix followed by either raw
binary or hex, depending on the option flag. OPTIONS_PRETTY_BINARY caused it
to emit "0x" before the hex representation of the data. This is wrong in
that it can't be read back by either the C++ or the Python LLSD parser.
Correct OPTIONS_PRETTY_BINARY formatting consists of b16"hex digits" rather
than b(len)"raw bytes".
- Although the code did set hex mode, it didn't set either the field width or
the fill character, so that a byte value less than 16 would emit a single
digit rather than two.
Instead of having one LLSDFormatter::format() method with an optional options
argument, declare two overloads. The format() overload without options passes
the mOptions data member to the overload accepting options.
Refactor the LLSDFormatter family, hoisting the recursive format_impl() method
(accepting level) to a pure virtual method at LLSDFormatter base-class level.
Most subclasses therefore need not override either base-class format() method,
only format_impl(). In fact the short format() overload isn't even virtual.
Consistently use LLSDFormatter::EFormatterOptions enum as the options
parameter wherever such options are accepted.
|
|
|
|
llmainthreadtask_test builds in a Sync timeout to keep build-time tests from
hanging. That timeout was set to 2000ms, which seems as though it ought to be
plenty enough time for a process with only 2 threads to exchange data between
them. But on TeamCity EC2 Windows build hosts, sometimes we hit that timeout
and fail. Extend it to try to improve the robustness of builds, even though
the possibility of a production viewer blocking for that long for anything
seems worrisome. (Fortunately the production viewer does not use Sync.)
|
|
|
|
|
|
Now that we have the Sync class to help construct unit tests that move forward
in a deterministic stepwise order, we can build suitable unit tests for
LLMainThreadTask.
|
|
llexception_test.cpp is about discovering appropriate infrastructure to get
good information from the LLTHROW() and LOG_UNHANDLED_EXCEPTION() mechanism.
But we didn't before have a test that actually exercises them. Now we do.
|
|
Introduce LLCoros::Stop exception, with subclasses Stopping, Stopped and
Shutdown. Add LLCoros::checkStop(), intended to be called periodically by any
coroutine with nontrivial lifespan. It checks the LLApp status and, unless
isRunning(), throws one of these new exceptions.
Make LLCoros::toplevel() catch Stop specially and log forcible coroutine
termination.
Now that LLApp status matters even in a test program, introduce a trivial
LLTestApp subclass whose sole function is to make isRunning() true.
(LLApp::setStatus() is protected: only a subclass can call it.) Add LLTestApp
instances to lleventcoro_test.cpp and lllogin_test.cpp.
Make LLCoros::toplevel() accept parameters by value rather than by const
reference so we can continue using them even after context switches.
Make private LLCoros::get_CoroData() static. Given that we've observed some
coroutines living past LLCoros destruction, making the caller call
LLCoros::instance() is more dangerous than encapsulating it within a static
method -- since the encapsulated call can check LLCoros::wasDeleted() first
and do something reasonable instead. This also eliminates the need for both a
const and non-const overload.
Defend LLCoros::delete_CoroData() (cleanup function for fiber_specific_ptr for
CoroData, implicitly called after coroutine termination) against calls after
~LLCoros().
Add a status string to coroutine-local data, with LLCoro::setStatus(),
getStatus() and RAII class TempStatus.
Add an optional 'when' string argument to LLCoros::printActiveCoroutines().
Make ~LLCoros() print the coroutines still active at destruction.
|
|
LLEventLogProxy can be introduced to serve as a logging proxy for an existing
LLEventPump subclass instance. Access through the LLEventLogProxy will be
logged; access directly to the underlying LLEventPump will not.
LLEventLogProxyFor<LLEventPumpSubclass> functions as a drop-in replacement for
the original LLEventPumpSubclass instance. It internally instantiates
LLEventPumpSubclass and serves as a proxy for that instance.
Add unit tests for LLEventMailDrop and LLEventLogProxyFor<LLEventMailDrop>,
both "plain" (events only) and via lleventcoro.h synchronization.
|
|
Sync is specifically intended for test programs. It is based on an
LLScalarCond<int>. The idea is that each of two coroutines can watch for the
other to get a chance to run, indicated by incrementing the wrapped int and
notifying the wrapped condition_variable. This is less hand-wavy than calling
llcoro::suspend() and hoping that the other routine will have had a chance to
run.
Use Sync in lleventcoro_test.cpp.
Also refactor lleventcoro_test.cpp so that instead of a collection of static
data requiring a clear() call at start of each individual test function, the
relevant data is all part of the test_data struct common to all test
functions. Make the helper coroutine functions members of test_data too.
Introduce llcoro::logname(), a convenience function to log the name of the
currently executing coroutine or "main" if in the thread's main coroutine.
|
|
If the test<1>() child process terminates with nonzero rc, also report any
stdout/stderr it might have emitted first.
|
|
|
|
|
|
build working
|
|
Longtime fans will remember that the "dcoroutine" library is a Google Summer
of Code project by Giovanni P. Deretta. He originally called it
"Boost.Coroutine," and we originally added it to our 3p-boost autobuild
package as such. But when the official Boost.Coroutine library came along
(with a very different API), and we still needed the API of the GSoC project,
we renamed the unofficial one "dcoroutine" to allow coexistence.
The "dcoroutine" library had an internal low-level API more or less analogous
to Boost.Context. We later introduced an implementation of that internal API
based on Boost.Context, a step towards eliminating the GSoC code in favor of
official, supported Boost code.
However, recent versions of Boost.Context no longer support the API on which
we built the shim for "dcoroutine." We started down the path of reimplementing
that shim using the current Boost.Context API -- then realized that it's time
to bite the bullet and replace the "dcoroutine" API with the Boost.Fiber API,
which we've been itching to do for literally years now.
Naturally, most of the heavy lifting is in llcoros.{h,cpp} and
lleventcoro.{h,cpp} -- which is good: the LLCoros layer abstracts away most of
the differences between "dcoroutine" and Boost.Fiber.
The one feature Boost.Fiber does not provide is the ability to forcibly
terminate some other fiber. Accordingly, disable LLCoros::kill() and
LLCoprocedureManager::shutdown(). The only known shutdown() call was in
LLCoprocedurePool's destructor.
We also took the opportunity to remove postAndSuspend2() and its associated
machinery: FutureListener2, LLErrorEvent, errorException(), errorLog(),
LLCoroEventPumps. All that dual-LLEventPump stuff was introduced at a time
when the Responder pattern was king, and we assumed we'd want to listen on one
LLEventPump with the success handler and on another with the error handler. We
have never actually used that in practice. Remove associated tests, of course.
There is one other semantic difference that necessitates patching a number of
tests: with "dcoroutine," fulfilling a future IMMEDIATELY resumes the waiting
coroutine. With Boost.Fiber, fulfilling a future merely marks the fiber as
ready to resume next time the scheduler gets around to it. To observe the test
side effects, we've inserted a number of llcoro::suspend() calls -- also in
the main loop.
For a long time we retained a single unit test exercising the raw "dcoroutine"
API. Remove that.
Eliminate llcoro_get_id.{h,cpp}, which provided llcoro::get_id(), which was a
hack to emulate fiber-local variables. Since Boost.Fiber has an actual API for
that, remove the hack.
In fact, use (new alias) LLCoros::local_ptr for LLSingleton's dependency
tracking in place of llcoro::get_id().
In CMake land, replace BOOST_COROUTINE_LIBRARY with BOOST_FIBER_LIBRARY. We
don't actually use the Boost.Coroutine for anything (though there exist
plausible use cases).
|
|
Given the viewer's mutually-dependent LLSingletons, given that different
threads might simultaneously request different LLSingletons from such a chain
of circular dependencies, the key to avoiding deadlock is to serialize all
LLSingleton construction on one thread: the main thread. Add comments to
LLSingleton::getInstance() explaining the problem and the solution.
Recast LLSingleton's static SingletonData to use LockStatic. Instead of using
Locker, and simply trusting that every reference to sData is within the
dynamic scope of a Locker instance, LockStatic enforces that: you can only
access SingletonData members via LockStatic.
Reorganize the switch in getInstance() to group the CONSTRUCTING error, the
INITIALIZING/INITIALIZED success case, and the DELETED/UNINITIALIZED
construction case.
When [re]constructing an instance, on the main thread, retain the lock and
call constructSingleton() (and capture_dependency()) directly.
On a secondary thread, unlock LockStatic and use LLMainThreadTask::dispatch()
to call getInstance() on the main thread. Since we might end up enqueuing
multiple such tasks, it's important to let getInstance() notice when the
instance has already been constructed and simply return the existing pointer.
Add loginfos() method, sibling to logerrs(), logwarns() and logdebugs().
Produce loginfos() messages when dispatching to the main thread, when actually
running on the main thread and when resuming the suspended requesting thread.
Make deleteSingleton() manage all associated state, instead of delegating some
of that work to ~LLSingleton(). Now, within LockStatic, extract the instance
pointer and set state to DELETED; that lets subsequent code, which retains the
only remaining pointer to the instance, remove the master-list entry, call the
subclass cleanupSingleton() and destructor without needing to hold the lock.
In fact, entirely remove ~LLSingleton().
Import LLSingletonBase::cleanup_() method to wrap the call to subclass
cleanupSingleton() in try/catch.
Remove cleanupAll() calls from llsingleton_test.cpp, and reorder the success
cases to reflect the fact that T::cleanupSingleton() is called immediately
before ~T() for each distinct LLSingleton subclass T.
When getInstance() on a secondary thread dispatches to the main thread, it
necessarily unlocks its LockStatic lock. But an LLSingleton dependency chain
strongly depends on the function stack on which getInstance() is invoked --
the task dispatched to the main thread doesn't know the dependencies tracked
on the requesting thread stack. So, once the main thread delivers the instance
pointer, the requesting thread captures its own dependencies for that
instance.
Back in the requesting thread, obtaining the current EInitState to pass to
capture_dependencies() would have required relocking LockStatic. Instead, I've
convinced myself that (a) capture_dependencies() only wanted to know
EInitState to produce an error for CONSTRUCTING, and (b) in CONSTRUCTING
state, we never get as far as capture_dependencies() because getInstance()
produces an error first.
Eliminate the EInitState parameter from all capture_dependencies() methods.
Remove the LLSingletonBase::capture_dependency() stanza that tested
EInitState. Make the capture_dependencies() variants that accepted LockStatic
instead accept LLSingletonBase*. That lets getInstance(), in the
LLMainThreadTask case, pass the newly-returned instance pointer.
For symmetry, make pop_initializing() accept LLSingletonBase* as well, instead
of accepting LockStatic and extracting mInstance.
|
|
The previous implementation went to some effort to crash if anyone attempted
to create or destroy an LLInstanceTracker subclass instance during traversal.
That restriction is manageable within a single thread, but becomes unworkable
if it's possible that a given subclass might be used on more than one thread.
Remove LLInstanceTracker::instance_iter, beginInstances(), endInstances(),
also key_iter, beginKeys() and endKeys(). Instead, introduce key_snapshot()
and instance_snapshot(), the only means of iterating over LLInstanceTracker
instances. (These are intended to resemble functions, but in fact the current
implementation simply presents the classes.) Iterating over a captured
snapshot defends against container modifications during traversal. The term
'snapshot' reminds the coder that a new instance created during traversal will
not be considered. To defend against instance deletion during traversal, a
snapshot stores std::weak_ptrs which it lazily dereferences, skipping on the
fly any that have expired.
Dereferencing instance_snapshot::iterator gets you a reference rather than a
pointer. Because some use cases want to delete all existing instances, add an
instance_snapshot::deleteAll() method that extracts the pointer. Those cases
used to require explicitly copying instance pointers into a separate
container; instance_snapshot() now takes care of that. It remains the caller's
responsibility to ensure that all instances of that LLInstanceTracker subclass
were allocated on the heap.
Replace unkeyed static LLInstanceTracker::getInstance(T*) -- which returned
nullptr if that instance had been destroyed -- with new getWeak() method
returning std::weak_ptr<T>. Caller must detect expiration of that weak_ptr.
Adjust tests accordingly.
Use of std::weak_ptr to detect expired instances requires engaging
std::shared_ptr in the constructor. We now store shared_ptrs in the static
containers (std::map for keyed, std::set for unkeyed).
Make LLInstanceTrackerBase a template parameterized on the type of the static
data it manages. For that reason, hoist static data class declarations out of
the class definitions to an LLInstanceTrackerStuff namespace.
Remove the static atomic sIterationNestDepth and its methods incrementDepth(),
decrementDepth() and getDepth(), since they were used only to forbid creation
and destruction during traversal.
Add a std::mutex to static data. Introduce an internal LockStatic class that
locks the mutex while providing a pointer to static data, making that the only
way to access the static data.
The LLINSTANCETRACKER_DTOR_NOEXCEPT macro goes away because we no longer
expect ~LLInstanceTracker() to throw an exception in test programs.
That affects LLTrace::StatBase as well as LLInstanceTracker itself.
Adapt consumers to the new LLInstanceTracker API.
|
|
|
|
|
|
This was forbidden, but AndreyK points out cases in which LLParamSingleton::
initSingleton() should in fact be allowed to circle back to its own instance()
method. Use a recursive_mutex instead of plain mutex to permit that; remove
LL_ERRS preventing it.
Add LLParamSingleton::instance() method that calls
LLParamSingleton::getInstance(). Inheriting LLSingleton::instance() called
LLSingleton::getInstance() -- not at all what we want.
Add LLParamSingleton unit tests.
|
|
|
|
Use them in place of awkward try/catch test boilerplate.
|
|
Use LLStringUtil::getenv() or getoptenv() whenever we fetch a string that will
be used as a pathname.
Use LLFile::tmpdir() instead of getenv("TEMP").
As an added extra-special bonus, finally clean up $TMP/llcontrol-test-zzzzzz
directories that have been accumulating every time we run a local build!
|
|
|
|
|
|
breaks
Improve the implementation so that escaping is computed only once
|
|
|
|
MAINT-8991: only escape log message characters once, add unit test
remove extra log line created by LL_ERRS
document that tags may not contain spaces
|
|
|
|
|
|
|
|
Streamline convenience overload stringize(std::wstring); make convenience
overload wstringize(std::string) symmetrically convert from UTF-8 string.
Also eliminate STRINGIZE() et al. dependency on Boost.Phoenix: use lambdas
instead.
Using lambdas instead of template expansion necessitates reordering some code
in wrapllerrs.h.
|
|
|
|
|
