- Modified the SandboxService and related app generators to replace workflow_execution_id with sandbox_id for improved clarity and consistency in sandbox handling.
- Adjusted the AdvancedChatAppGenerator and WorkflowAppGenerator to align with the new parameter naming convention.
- Updated API routes to use app_id instead of sandbox_id for file operations, aligning with user-specific sandbox workspaces.
- Enhanced SandboxFileService and related classes to accommodate app_id in file listing and download functionalities.
- Refactored storage key generation for sandbox archives to include app_id, ensuring proper file organization.
- Adjusted frontend contracts and services to reflect the new app_id parameter in API calls.
- Updated the AWS S3 client initialization to enforce SigV4 for presigned URLs, ensuring consistent header signing behavior across S3-compatible providers.
- Introduced a dedicated configuration for the S3 client to manage addressing styles and signature versions more effectively.
- Implemented the `exists` method in `SandboxFileSource` and its subclasses to verify the availability of sandbox sources.
- Updated `SandboxFileService` to utilize the new `exists` method for improved error handling when listing files and downloading files.
- Removed the previous check for storage existence in `archive_source.py` and replaced it with the new method.
The backend part of the human in the loop (HITL) feature and relevant architecture / workflow engine changes.
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: 盐粒 Yanli <yanli@dify.ai>
Co-authored-by: CrabSAMA <40541269+CrabSAMA@users.noreply.github.com>
Co-authored-by: Stephen Zhou <38493346+hyoban@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
- Replaced direct storage references with SilentStorage in various components to enhance fallback mechanisms.
- Updated storage key formats for sandbox archives and files to improve clarity and consistency.
- Refactored related classes and methods to utilize the new SandboxFilePath structure.
- Adjusted unit tests to reflect changes in the StorageTicket model and its serialization methods.
- Removed the storage proxy controller and its associated endpoints for file download and upload.
- Updated the file controller to use the new storage ticket service for generating download and upload URLs.
- Modified the file presign storage to fallback to ticket-based URLs instead of signed proxy URLs.
- Enhanced unit tests to validate the new ticket generation and retrieval logic.
- Removed unused app asset download and upload endpoints, along with sandbox archive and file download endpoints.
- Updated imports in the file controller to reflect the removal of these endpoints.
- Simplified the generator.py file by consolidating the code context field definition.
- Enhanced the storage layer with a unified presign wrapper for better handling of presigned URLs.
- Moved the import of Queue and Empty from the top of the file to within the QueueTransportReadCloser class.
- This change improves encapsulation and ensures that the imports are only available where needed.
- Add BundleManifest with dsl_filename for 100% tree ID restoration
- Implement two-step import flow: prepare (get upload URL) + confirm
- Use sandbox for zip extraction and file upload via presigned URLs
- Store import session in Redis with 1h TTL
- Add SandboxUploadItem for symmetric download/upload API
- Remove legacy source_zip_extractor, inline logic in service
- Update frontend to use new prepare/confirm API flow
- Added an `enabled` field to `DifyCliToolConfig` and `ToolDependency` to manage tool activation status.
- Updated `DifyCliConfig` to handle tool dependencies more effectively, ensuring only enabled tools are processed.
- Refactored `SkillCompiler` to utilize `tool_id` for better identification of tools and improved handling of disabled tools.
- Introduced a new method `_extract_disabled_tools` in `LLMNode` to streamline the extraction of disabled tools from node data.
- Enhanced metadata parsing to account for tool enablement, improving overall tool management.
- Introduced a new regex pattern for tool groups to support multiple tool placeholders.
- Updated the DefaultToolResolver to format outputs for specific built-in tools (bash, python).
- Enhanced the SkillCompiler to filter out disabled tools in tool groups, ensuring only enabled tools are rendered.
- Added tests to verify the correct behavior of tool group filtering and rendering.
- Introduced a new boolean field `computer_use` in LLMNodeData to indicate whether the computer use feature should be enabled.
- Updated LLMNode to check the `computer_use` field when determining sandbox usage, ensuring proper error handling if sandbox is not available.
- Removed the obsolete `_has_skill_prompt` method to streamline the code.
Updated multiple modules to utilize gevent for concurrency, ensuring compatibility with gevent-based WSGI servers. This includes replacing threading.Thread and threading.Event with gevent.spawn and gevent.event.Event, respectively, to prevent blocking and improve performance during I/O operations.
- Refactored SandboxBuilder, Sandbox, CommandFuture, and DockerDemuxer to use gevent.
- Added detailed docstrings explaining the changes and benefits of using gevent primitives.
This change enhances the responsiveness and efficiency of the application in a gevent environment.
- Added sandbox archive upload and download proxy endpoints with signed URL verification.
- Introduced security helpers for generating and verifying signed URLs.
- Updated file-related API routes to include sandbox archive functionality.
- Refactored app asset storage methods to streamline download/upload URL generation.
- Added a threaded function to handle the deletion of storage files asynchronously after asset removal.
- Updated the asset removal logic to include a call to the new deletion function, improving performance and responsiveness during asset management operations.
- Updated the download script to redirect error output to /dev/null, preventing unnecessary error messages from being displayed.
- Added an explicit exit command to ensure the script terminates correctly after execution.
- Updated storage wrappers to utilize a new base class, StorageWrapper, for better delegation of methods.
- Introduced SilentStorage to handle read operations gracefully by returning empty values instead of raising exceptions.
- Enhanced CachedPresignStorage to support batch caching of download URLs, improving performance.
- Refactored FilePresignStorage to support both presigned URLs and signed proxy URLs for downloads.
- Updated AppAssetService to utilize the new storage structure, ensuring consistent asset management.
- Introduced CachedPresignStorage to cache presigned download URLs, reducing repeated API calls.
- Updated AppAssetService to utilize CachedPresignStorage for improved performance in asset download URL generation.
- Refactored asset builders and packagers to support the new storage mechanism.
- Removed unused AppAssetsAttrsInitializer to streamline initialization processes.
- Added unit tests for CachedPresignStorage to ensure functionality and reliability.
The `/api/system-features` is required for the web app initialization.
Authentication would create circular dependency (can't authenticate without web app loading).
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
- Deleted `AppAssetFileResource` class and its associated file upload logic.
- Removed the `create_file` method from `AppAssetService` to streamline asset management.
- Updated `AppAssetBatchUploadResource` for improved readability by condensing method calls.
- Added `RuntimeType` enum to define app runtime types: CLASSIC and SANDBOXED.
- Updated `AppPartial` model to include `runtime_type` field.
- Enhanced `AppListApi` to determine and assign the appropriate runtime type based on sandbox feature availability.
- Introduced `GetUploadUrlPayload` and `BatchUploadPayload` models for handling file uploads.
- Implemented `AppAssetFileUploadUrlResource` for generating pre-signed upload URLs.
- Added `AppAssetBatchUploadResource` to support batch creation of asset nodes from a tree structure.
- Enhanced `AppAssetService` with methods for obtaining upload URLs and batch creation of assets.
- Removed checksum handling from file creation to streamline the process.
- Updated the `get_system_default_config` method to accept a `provider_type` parameter for more precise querying.
- Improved error handling to raise a ValueError if no system default provider is configured for the specified tenant and provider type.
- Added fallback logic to ensure a system default configuration is returned when available.
- Updated the SandboxManager to rename the method for deleting storage to better reflect its purpose.
- Adjusted the WorkflowVariableCollectionApi to utilize the new method name.
- Improved error handling in ArchiveSandboxStorage's delete method to log exceptions during deletion.
- Renamed `build_skill_artifact_set` to `build_skill_bundle` for improved clarity in asset management.
- Updated references in `SkillManager` to reflect the new method name and ensure consistent handling of skill bundles.
- Added `AppAssetsAttrsInitializer` to `SandboxManager` to enhance asset initialization processes.
- Implemented output truncation in `SandboxBashTool` to manage long command outputs effectively.
- Introduced AppBundleService for managing app bundle publishing and importing, integrating workflow and asset services.
- Added methods for exporting app bundles as ZIP files, including DSL and asset management.
- Implemented source zip extraction and validation to enhance asset import processes.
- Refactored asset packaging to utilize AssetZipPackager for improved performance and organization.
- Enhanced error handling for bundle format and security during import operations.
- Replaced SkillArtifactSet with SkillBundle across various components, enhancing the organization of skill dependencies and references.
- Updated SkillManager methods to load and save bundles instead of artifacts, improving clarity in asset management.
- Refactored SkillCompiler to compile skills into bundles, streamlining the dependency resolution process.
- Adjusted DifyCli and SandboxBashSession to utilize ToolDependencies, ensuring consistent handling of tool references.
- Introduced AssetReferences for better management of file dependencies within skill bundles.
Stream
Asuka Minato
盐粒 Yanli
Harry
zhsama
lif
yyh
QuantumGhost
Cursx
Jax
Novice
FFXN
hjlarry
yihong
-LAN-
Nie Ronghua
eux
fenglin
moonpanda
NeatGuyCoding
heyszt
hj24
dependabot[bot]
E.G
wangxiaolei
Junyan Qin (Chin)
Joel
Xiangxuan Qu
zyssyz123
autofix-ci[bot]
TomoOkuyama
Yeuoly