From 03c4458bdcc6821a3047f93b729d412e274ab9af Mon Sep 17 00:00:00 2001 From: Dave Parks Date: Mon, 20 May 2024 13:22:55 -0500 Subject: #1392 GLTF Upload (#1394) * #1392 WIP -- Functional texture upload, stubbed out .bin upload. * #1392 GLTF Upload WIP -- Emulates successful upload Successfully uploads texture Emulates successful .gltf and .bin upload by injecting into local asset cache. Emulates rez from inventory by setting sculpt ID of selected object Currently fails in tinygltf parsing due to missing .bin * Add missing notification * Build fix * #1392 Add boost::json .gltf reading support. * #1392 boost::json GLTF writing prototype * Create gltf/README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * #1392 Add ability to render directly from LL::GLTF::Material * Fix for mac build * Mac build fix * #1392 AssetType and Inventory Type plumbing * #1392 More sane error handling and scheduling of uploads. * #1392 Actually attempt to upload glbin * Mac build fix, upload nudge * Mac build fix * Fix glTF asset uploads to server * Mac build fix (inline not static) * More consistent inline * Add glm, mac nudge. * #1392 For consistency with spec, start using glm over glh:: and LLFoo * Another attempt at placating Mac builds * Another Mac nudge * Mac build take 23 * #1392 Prune LLMatrix4a from GLTF namespace. * #1392 Fix for orientation being off (glm::quat is wxyz, not xyzw) * #1392 WIP -- Actually send the sculpt type and id, nudge readme and alpha rendering * #1392 Working download! * #1394 Add support for GLTFEnabled SimulatorFeature * #1392 Review feedback --------- Co-authored-by: Pepper Linden <3782201+rohvani@users.noreply.github.com> --- indra/newview/gltf/README.md | 156 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 indra/newview/gltf/README.md (limited to 'indra/newview/gltf/README.md') diff --git a/indra/newview/gltf/README.md b/indra/newview/gltf/README.md new file mode 100644 index 0000000000..8e7df0a439 --- /dev/null +++ b/indra/newview/gltf/README.md @@ -0,0 +1,156 @@ +# Linden Lab GLTF Implementation + +Currently in prototype stage. Much functionality is missing (blend shapes, +multiple texture coordinates, etc). + +GLTF Specification can be found here: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html. +If this implementation disagrees with the GLTF Specification, the specification is correct. + +Class structure and naming should match the GLTF Specification as closely as possible while +conforming to the LL coding standards. All code in headers should be contained in the +LL::GLTF namespace. + +The implementation serves both the client and the server. + +## Design Principles + +- The implementation MUST be capable of round-trip serialization with no data loss beyond F64 to F32 conversions. +- The implementation MUST use the same indexing scheme as the GLTF specification. Do not store pointers where the +- GLTF specification stores indices, store indices. +- Limit dependencies on llcommon as much as possible. Prefer std::, boost::, and (soon) glm:: over LL facsimiles. +- Usage of LLSD is forbidden in the LL::GLTF namespace. +- Use "using namespace" liberally in .cpp files, but never in .h files. +- "using Foo = Bar" is permissible in .h files within the LL::GLTF namespace. + +## Loading, Copying, and Serialization +Each class should provide two functions (Primitive shown for example): + +``` +// Serialize to the provided json object. +// "obj" should be "this" in json form on return +// Do not serialize default values +void serialize(boost::json::object& obj) const; + +// Initialize from a provided json value +const Primitive& operator=(const Value& src); +``` + +"serialize" implementations should use "write": + +``` +void Primitive::serialize(boost::json::object& dst) const +{ + write(mMaterial, "material", dst, -1); + write(mMode, "mode", dst, TINYGLTF_MODE_TRIANGLES); + write(mIndices, "indices", dst, INVALID_INDEX); + write(mAttributes, "attributes", dst); +} +``` + +And operator= implementations should use "copy": + +``` +const Primitive& Primitive::operator=(const Value& src) +{ + if (src.is_object()) + { + copy(src, "material", mMaterial); + copy(src, "mode", mMode); + copy(src, "indices", mIndices); + copy(src, "attributes", mAttributes); + + mGLMode = gltf_mode_to_gl_mode(mMode); + } + return *this; +} +``` + +Parameters to "write" and "copy" MUST be ordered "src" before "dst" +so the code reads as "write src to dst" and "copy src to dst". + +When reading string constants from GLTF json (i.e. "OPAQUE", "TRIANGLES"), these +strings should be converted to enums inside operator=. It is permissible to +store the original strings during prototyping to aid in development, but eventually +we'll purge these strings from the implementation. However, implementations MUST +preserve any and all "name" members. + +"write" and "copy" implementations MUST be stored in buffer_util.h. +As implementers encounter new data types, you'll see compiler errors +pointing at templates in buffer_util.h. See vec3 as a known good +example of how to add support for a new type (there are bad examples, so beware): + +``` +// vec3 +template<> +inline bool copy(const Value& src, vec3& dst) +{ + if (src.is_array()) + { + const boost::json::array& arr = src.as_array(); + if (arr.size() == 3) + { + if (arr[0].is_double() && + arr[1].is_double() && + arr[2].is_double()) + { + dst = vec3(arr[0].get_double(), arr[1].get_double(), arr[2].get_double()); + } + return true; + } + } + return false; +} + +template<> +inline bool write(const vec3& src, Value& dst) +{ + dst = boost::json::array(); + boost::json::array& arr = dst.as_array(); + arr.resize(3); + arr[0] = src.x; + arr[1] = src.y; + arr[2] = src.z; + return true; +} + +``` + +"write" MUST return true if ANY data was written +"copy" MUST return true if ANY data was copied + +Speed is important, but so is safety. In writers, try to avoid redundant copies +(prefer resize over push_back, convert dst to an empty array and fill it, don't +make an array on the stack and copy it into dst). + +boost::json WILL throw exceptions if you call as_foo() on a mismatched type but +WILL NOT throw exceptions on get_foo with a mismatched type. ALWAYS check is_foo +before calling as_foo or get_foo. DO NOT add exception handlers. If boost throws +an exception in serialization, the fix is to add type checks. If we see a large +number of crash reports from boost::json exceptions, each of those reports +indicates a place where we're missing "is_foo" checks. They are gold. Do not +bury them with an exception handler. + +DO NOT rely on existing type conversion tools in the LL codebase -- LL data models +conflict with the GLTF specification so we MUST provide conversions independent of +our existing implementations. + +### JSON Serialization ### + + + +NEVER include buffer_util.h from a header. + +Loading from and saving to disk (import/export) is currently done using tinygltf, but this is not a long term +solution. Eventually the implementation should rely solely on boost::json for reading and writing .gltf +files and should handle .bin files natively. + +When serializing Images and Buffers to the server, clients MUST store a single UUID "uri" field and nothing else. +The server MUST reject any data that violates this requirement. + +Clients MUST remove any Images from Buffers prior to upload to the server. +Servers MAY reject Assets that contain Buffers with unreferenced data. + +... to be continued. + + + -- cgit v1.2.3 From 2f4120038429c6aff865f153f708ceefb60d67f4 Mon Sep 17 00:00:00 2001 From: Dave Parks Date: Tue, 28 May 2024 09:45:40 -0500 Subject: Remove tinygltf dependency from LL::GLTF (#1541) * #1535 Image loading/saving support in boost::json driven GLTF parser * #1536 GLB Support in boost::json drvien GLTF parser --- indra/newview/gltf/README.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) (limited to 'indra/newview/gltf/README.md') diff --git a/indra/newview/gltf/README.md b/indra/newview/gltf/README.md index 8e7df0a439..a2d43be1d6 100644 --- a/indra/newview/gltf/README.md +++ b/indra/newview/gltf/README.md @@ -1,13 +1,13 @@ # Linden Lab GLTF Implementation - -Currently in prototype stage. Much functionality is missing (blend shapes, + +Currently in prototype stage. Much functionality is missing (blend shapes, multiple texture coordinates, etc). GLTF Specification can be found here: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html. If this implementation disagrees with the GLTF Specification, the specification is correct. Class structure and naming should match the GLTF Specification as closely as possible while -conforming to the LL coding standards. All code in headers should be contained in the +conforming to the LL coding standards. All code in headers should be contained in the LL::GLTF namespace. The implementation serves both the client and the server. @@ -18,7 +18,7 @@ The implementation serves both the client and the server. - The implementation MUST use the same indexing scheme as the GLTF specification. Do not store pointers where the - GLTF specification stores indices, store indices. - Limit dependencies on llcommon as much as possible. Prefer std::, boost::, and (soon) glm:: over LL facsimiles. -- Usage of LLSD is forbidden in the LL::GLTF namespace. +- Usage of LLSD is forbidden in the LL::GLTF namespace. - Use "using namespace" liberally in .cpp files, but never in .h files. - "using Foo = Bar" is permissible in .h files within the LL::GLTF namespace. @@ -69,14 +69,14 @@ Parameters to "write" and "copy" MUST be ordered "src" before "dst" so the code reads as "write src to dst" and "copy src to dst". When reading string constants from GLTF json (i.e. "OPAQUE", "TRIANGLES"), these -strings should be converted to enums inside operator=. It is permissible to +strings should be converted to enums inside operator=. It is permissible to store the original strings during prototyping to aid in development, but eventually we'll purge these strings from the implementation. However, implementations MUST preserve any and all "name" members. "write" and "copy" implementations MUST be stored in buffer_util.h. As implementers encounter new data types, you'll see compiler errors -pointing at templates in buffer_util.h. See vec3 as a known good +pointing at templates in buffer_util.h. See vec3 as a known good example of how to add support for a new type (there are bad examples, so beware): ``` @@ -120,29 +120,29 @@ inline bool write(const vec3& src, Value& dst) Speed is important, but so is safety. In writers, try to avoid redundant copies (prefer resize over push_back, convert dst to an empty array and fill it, don't -make an array on the stack and copy it into dst). +make an array on the stack and copy it into dst). -boost::json WILL throw exceptions if you call as_foo() on a mismatched type but -WILL NOT throw exceptions on get_foo with a mismatched type. ALWAYS check is_foo +boost::json WILL throw exceptions if you call as_foo() on a mismatched type but +WILL NOT throw exceptions on get_foo with a mismatched type. ALWAYS check is_foo before calling as_foo or get_foo. DO NOT add exception handlers. If boost throws an exception in serialization, the fix is to add type checks. If we see a large number of crash reports from boost::json exceptions, each of those reports -indicates a place where we're missing "is_foo" checks. They are gold. Do not +indicates a place where we're missing "is_foo" checks. They are gold. Do not bury them with an exception handler. -DO NOT rely on existing type conversion tools in the LL codebase -- LL data models -conflict with the GLTF specification so we MUST provide conversions independent of +DO NOT rely on existing type conversion tools in the LL codebase -- LL data models +conflict with the GLTF specification so we MUST provide conversions independent of our existing implementations. ### JSON Serialization ### -NEVER include buffer_util.h from a header. +NEVER include buffer_util.h from a header. Loading from and saving to disk (import/export) is currently done using tinygltf, but this is not a long term solution. Eventually the implementation should rely solely on boost::json for reading and writing .gltf -files and should handle .bin files natively. +files and should handle .bin files natively. When serializing Images and Buffers to the server, clients MUST store a single UUID "uri" field and nothing else. The server MUST reject any data that violates this requirement. @@ -152,5 +152,5 @@ Servers MAY reject Assets that contain Buffers with unreferenced data. ... to be continued. - + -- cgit v1.2.3