50 Commits

Author SHA1 Message Date
zhouhao
e395e4041e build: 升级依赖 2025-07-23 10:06:47 +08:00
zhouhao
e134dfec6f build: 升级依赖版本 2025-06-30 13:54:01 +08:00
zhouhao
3a45ac86da refactor: 升级依赖 2025-06-19 16:05:08 +08:00
zhouhao
f99b648b88 refactor: 优化兼容性 2025-06-12 14:05:48 +08:00
zhouhao
d4645ad089 refactor: 优化查询条件转换 2025-06-03 19:14:08 +08:00
zhouhao
88e0dd4667 Merge remote-tracking branch 'origin/master' 2025-05-27 10:41:27 +08:00
zhouhao
1bd136f3d2 refactor: 移除无用统计 2025-05-27 10:41:14 +08:00
Zhang Ji
fc651ecd9d fix(场景联动): 修复执行动作的内置参数获取可能失效的问题 (#636) 2025-05-26 19:03:49 +08:00
zhouhao
dbceadc5e0 Merge remote-tracking branch 'origin/master' 2025-05-20 15:32:09 +08:00
zhouhao
520ca24fed refactor: 增加ui resource接口 2025-05-20 15:31:49 +08:00
laokou
a6100d36f9 fix: 修复MqttClient网络组件定义的元数据名称和类型错误 (#632) 2025-05-06 09:25:11 +08:00
zhouhao
80ed50211a refactor: 优化标签存储 2025-04-29 18:11:00 +08:00
zhouhao
3a1f0b65de fix: 修复设备标签无法更新问题 2025-04-28 18:33:07 +08:00
zhouhao
2cfc78f1da Merge remote-tracking branch 'origin/master' 2025-04-28 11:41:26 +08:00
zhouhao
ad34196a2a refactor: mqtt服务接入支持onClientConnect 2025-04-28 11:41:08 +08:00
老周
1e37c685cc Update README.md 2025-04-27 18:12:06 +08:00
老周
7fdfe87e9d build: 2.3 版本发布 (#615)
* build: 2.3 版本发布

* feat: 更新网关及设备管理相关模块代码 (#614)

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* feat(community): 场景联动以及通知模块2.3相关功能更新 (#612)

* feat(community): 场景联动以及通知模块2.3相关功能更新

* Update pull_request.yml

* feat(community): 场景联动清除资产相关代码

* fix(场景联动): 修复场景初始化问题

* feat(community): 注册个人订阅通道

* fix(通知模块): 订阅通道初始化

* fix(通知模块): 删除无用类

* fix(community): 解决合并冲突

---------

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>
Co-authored-by: 老周 <zh.sqy@qq.com>

* build: 升级依赖版本

* feat(community): 认证模块更新2.3相关功能 (#616)

* feat(community): 认证模块更新2.3相关功能

* fix(community): 修复自定义查询报错的问题

* fix(community): 优化邮件附件名称获取

---------

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* feat(community): 补充国际化相关内容,修复设备上线/离线没有日志的问题 (#619)

* feat(community): 补充国际化相关内容,修复设备上线/离线没有日志的问题

* feat(community): 补充设备物模型映射接口

---------

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* fix(notify): 修复消息通知相关问题 (#620)

* fix(notify): 修复消息通知相关问题

* fix(notify): 修复订阅管理数据展示错误问题

---------

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* fix(rule-engine): 场景联动相关bug修复 (#621)

* fix(rule-engine): 场景联动相关bug修复

* fix(rule-engine): 场景联动相关bug修复

* fix(rule-engine): 场景联动相关bug修复

* fix(rule-engine): 修复程序启动时订阅提供商禁用,但用户已订阅仍加载成功并订阅的问题

---------

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* fix(community): 修复认证中心以及设备相关的bug (#622)

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* feat(auth): 增加密码校验功能 (#625)

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* fix(auth): 修复组织父节点清除失败问题 (#627)

* fix(auth): 修复组织父节点清除失败问题

* fix(store): 补充存储模式名称国际化方法

---------

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* fix(auth): 修复用户绑定组织,查询用户关联组织失败的问题 (#628)

Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>

* build: 升级依赖版本

---------

Co-authored-by: fighter-wang <118291973+fighter-wang@users.noreply.github.com>
Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>
2025-04-25 19:15:32 +08:00
老周
08a358ffab Update pull_request.yml 2025-03-28 09:52:51 +08:00
zhouhao
0325eaddbf refactor: 优化设备注册逻辑 2025-03-26 14:46:10 +08:00
zhouhao
f640e2b45f refactor: 优化会话处理逻辑 2025-03-26 11:06:02 +08:00
zhouhao
091b8c4f0c fix: 修复编译错误 2025-03-24 10:13:21 +08:00
zhouhao
d3adf0e752 refactor: 优化设备会话持久化逻辑 2025-03-24 09:53:50 +08:00
zhouhao
7b7b96f096 refactor: 优化DeviceGatewayHelper 2025-03-19 10:54:27 +08:00
zhouhao
c01d31606c refactor: 优化资源释放 2025-03-13 10:25:32 +08:00
老周
fd2d73d4eb Update README.md 2025-02-17 09:12:45 +08:00
老周
2e487a42e2 feat(基础模块): 增加命令模式支持 (#607)
可使用 CommandSupportManagerProviders相关方法来执行服务命令进行解耦.
2025-02-13 12:33:18 +08:00
bestfeng1020
694595d446 fix(docker配置): 修复文件路径挂载错误 (#602)
* fix(docker配置): 修复文件路径挂载错误

* Update docker-compose.yml

---------

Co-authored-by: 老周 <zh.sqy@qq.com>
2025-02-12 11:52:22 +08:00
Zhang Ji
27a9fd6618 feat(场景联动): 添加设备数据执行动作,扩展数组条件,优化国际化 (#605) 2025-01-23 18:57:15 +08:00
bestfeng1020
ad6279ed22 feat(readme): 添加新QQ群信息 (#604)
feat(readme): 添加新QQ群信息
2025-01-13 15:29:35 +08:00
bestfeng1020
bc0c01d01f fix(通用模块): 修复TDengine排序逻辑 (#599) 2024-12-31 10:35:05 +08:00
fighter-wang
6e40f6fb8a fix(基础模块): 修复阿里云语音仅拨打一个用户号码的问题 (#597)
Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>
2024-12-18 16:15:42 +08:00
zhouhao
647f6ec9f3 fix(基础模块): 修复mqtt client 设备会话恢复后gatewayId为null问题 2024-12-16 17:52:17 +08:00
老周
7d79927d4d Merge pull request #594 from zxl1951/fix-td-save
fix(存储策略): 涛思数据库存储使用无模式写入创建表缺失messageId列
2024-12-11 19:36:27 +08:00
ZxL
806db31c4a fix(存储策略): 涛思数据库存储使用无模式写入创建表缺失messageId列 2024-12-11 17:17:36 +08:00
老周
cd6f2f1049 Merge pull request #593 from jetlinks/bestfeng1020-patch-1
feat(readme): 删除DTU售卖信息
2024-12-11 10:06:40 +08:00
bestfeng1020
89eb2d66a8 feat(readme): 删除DTU售卖信息
feat(readme): 删除DTU售卖信息
2024-12-11 09:53:49 +08:00
fighter-wang
9c26d7d6c3 feat(设备管理): 新增在reactorQL中获取设备属性上报时间 (#592)
Co-authored-by: fighter-wang <11291691+fighter-wang@user.noreply.gitee.com>
2024-12-09 13:53:14 +08:00
fighter-wang
b890f4e520 fix(关系配置): 关系配置增加反转关系名称字段 (#591) 2024-12-05 19:50:57 +08:00
zhouhao
f2bed57dc6 Merge remote-tracking branch 'origin/master' 2024-12-05 16:57:17 +08:00
zhouhao
13dd4e5e40 refactor: 优化mqtt 客户端配置 2024-12-05 16:56:59 +08:00
bestfeng1020
13ea87ff46 feat(docker镜像配置): 修改docker镜像版本 (#590) 2024-12-03 19:40:46 +08:00
zhouhao
d2772e8881 refactor: 优化列式存储策略数字类型转换逻辑 2024-11-06 15:50:59 +08:00
zhouhao
b5944c6d3a fix: #580 2024-10-29 15:02:06 +08:00
zhouhao
77abf004f7 Merge remote-tracking branch 'origin/master' 2024-10-21 17:26:09 +08:00
zhouhao
945bf2ea82 refactor(基础模块): 优化设备连接 2024-10-21 17:25:56 +08:00
dependabot[bot]
b83dc75943 build(deps): bump commons-io:commons-io from 2.11.0 to 2.15.1 in /jetlinks-components/io-component (#576)
* build(deps): bump commons-io:commons-io

Bumps commons-io:commons-io from 2.11.0 to 2.14.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pom.xml

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 老周 <zh.sqy@qq.com>
2024-10-08 13:32:20 +08:00
dependabot[bot]
cc55f4ad6c build(deps): bump org.elasticsearch:elasticsearch (#546)
Bumps [org.elasticsearch:elasticsearch](https://github.com/elastic/elasticsearch) from 7.17.13 to 7.17.23.
- [Release notes](https://github.com/elastic/elasticsearch/releases)
- [Changelog](https://github.com/elastic/elasticsearch/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elastic/elasticsearch/compare/v7.17.13...v7.17.23)

---
updated-dependencies:
- dependency-name: org.elasticsearch:elasticsearch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 09:41:40 +08:00
XiXuanHao
9b9545ecca fix(产品管理): 产品启用时会重复触发DeviceProductDeployEvent事件 (#513)
Co-authored-by: XIXUANHAO <xi_xh@foxmail.com>
2024-10-08 09:40:17 +08:00
zhouhao
6dc16ce715 Merge remote-tracking branch 'origin/master' 2024-09-27 11:09:34 +08:00
zhouhao
e3d7831073 build(maven): 2.3.0-SNAPSHOT 2024-09-27 11:09:22 +08:00
531 changed files with 20786 additions and 4274 deletions

View File

@@ -19,7 +19,7 @@ jobs:
with:
java-version: 1.8
- name: Cache Maven Repository
uses: actions/cache@v1
uses: actions/cache@v4.2.3
with:
path: ~/.m2
key: jetlinks-community-maven-repository

View File

@@ -1,12 +1,13 @@
# JetLinks 物联网基础平台
![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/jetlinks/jetlinks-community/maven.yml?branch=master)
![Version](https://img.shields.io/badge/version-2.1--RELEASE-brightgreen)
![Version](https://img.shields.io/badge/version-2.3-brightgreen)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/e8d527d692c24633aba4f869c1c5d6ad)](https://app.codacy.com/gh/jetlinks/jetlinks-community?utm_source=github.com&utm_medium=referral&utm_content=jetlinks/jetlinks-community&utm_campaign=Badge_Grade_Settings)
[![OSCS Status](https://www.oscs1024.com/platform/badge/jetlinks/jetlinks-community.svg?size=small)](https://www.oscs1024.com/project/jetlinks/jetlinks-community?ref=badge_small)
[![star](https://img.shields.io/github/stars/jetlinks/jetlinks-community?style=social)](https://github.com/jetlinks/jetlinks-community)
[![star](https://gitee.com/jetlinks/jetlinks-community/badge/star.svg?theme=gvp)](https://gitee.com/jetlinks/jetlinks-community/stargazers)
[![QQ⑥群572077464](https://img.shields.io/badge/QQ⑥群-572077464-brightgreen)](https://qm.qq.com/q/kLT3trlXuE)
[![QQ⑤群554591908](https://img.shields.io/badge/QQ⑤群-554591908-brightgreen)](https://qm.qq.com/cgi-bin/qm/qr?k=jiirLiyFUecy_gsankzVQ-cl6SrZCnv9&&jump_from=webapi)
[![QQ④群780133058](https://img.shields.io/badge/QQ④群-780133058-brightgreen)](https://qm.qq.com/cgi-bin/qm/qr?k=Gj47w9kg7TlV5ceD5Bqew_M_O0PIjh_l&jump_from=webapi)
[![QQ③群647954464](https://img.shields.io/badge/QQ③群-647954464-brightgreen)](https://qm.qq.com/cgi-bin/qm/qr?k=K5m27CkhDn3B_Owr-g6rfiTBC5DKEY59&jump_from=webapi)
@@ -104,7 +105,6 @@ TCP/UDP/MQTT/HTTP、TLS/DTLS、不同厂商、不同设备、不同报文、统
| 系统部署 | 在客户指定的网络和硬件环境中完成社区版服务部署;提供**模拟**设备接入到平台中,并能完成正常设备上线、数据上下行 | 199元 | 线上部署支持 |
| 技术支持 | 提供各类部署、功能使用中遇到的问题答疑 | 100元 | 半小时内 线上远程支持|
| 设备接入协议开发 | 根据提供的设备型号,编写并提供接入平台协议包的源码。| 3000+元 | 定制化开发 |
| 硬件支持 | 提供JetLinks自有硬件邮寄到手JetLinks DTU、报警器、温度传感器并提供完整的接入视频文档 | 699元 | 硬件包邮<br /> [视频文档](https://hanta.yuque.com/px7kg1/yfac2l/pgi0eydsmlyb7q1w)|
| 其他服务 | 企业版源码购买;定制化开发;定制化时长、功能服务等 | 面议 | 面议 |
### **付费**服务支持或商务合作请联系

View File

@@ -48,7 +48,7 @@ services:
POSTGRES_DB: jetlinks
TZ: Asia/Shanghai
ui:
image: registry.cn-shenzhen.aliyuncs.com/jetlinks/jetlinks-ui-vue:2.2.0-SNAPSHOT
image: registry.cn-shenzhen.aliyuncs.com/jetlinks/jetlinks-ui-vue:2.3.0-SNAPSHOT
container_name: jetlinks-ce-ui
ports:
- 9000:80
@@ -59,7 +59,7 @@ services:
links:
- jetlinks:jetlinks
jetlinks:
image: registry.cn-shenzhen.aliyuncs.com/jetlinks/jetlinks-community:2.2.0-SNAPSHOT
image: registry.cn-shenzhen.aliyuncs.com/jetlinks/jetlinks-community:2.3.0-SNAPSHOT
container_name: jetlinks-ce
ports:
@@ -68,10 +68,8 @@ services:
- "8800-8810:8800-8810" # 预留
- "5060-5061:5060-5061" # 预留
volumes:
- "./data/jetlinks:/application/static/upload" # 持久化上传的文件
- "./data/jetlinks/:/application/data/files"
- "./data/jetlinks/:/application/data/protocols"
- "./entrypoint.sh:/entrypoint.sh"
- "./data/jetlinks/upload:/application/static/upload"
- "./data/jetlinks:/application/data"
#entrypoint: /entrypoint.sh -d redis:5601,postgres:5432,elasticsearch:9200 'echo "start jetlinks service here"';
environment:
# - "SLEEP_SECOND=4"
@@ -115,4 +113,4 @@ services:
depends_on:
- postgres
- redis
- elasticsearch
- elasticsearch

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>jetlinks-components</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
@@ -19,6 +19,11 @@
<version>${jetlinks.version}</version>
</dependency>
<dependency>
<groupId>org.jetlinks</groupId>
<artifactId>jetlinks-supports</artifactId>
</dependency>
<dependency>
<groupId>org.hswebframework.web</groupId>
<artifactId>hsweb-authorization-api</artifactId>

View File

@@ -1,6 +1,9 @@
package org.jetlinks.community;
import org.jetlinks.core.config.ConfigKey;
import org.jetlinks.core.metadata.MergeOption;
import java.util.Map;
/**
* 数据验证配置常量类
@@ -20,4 +23,8 @@ public interface ConfigMetadataConstants {
ConfigKey<String> format = ConfigKey.of("format", "格式", String.class);
ConfigKey<String> defaultValue = ConfigKey.of("defaultValue", "默认值", String.class);
ConfigKey<Boolean> indexEnabled = ConfigKey.of("indexEnabled", "开启索引", Boolean.TYPE);
}

View File

@@ -0,0 +1,16 @@
package org.jetlinks.community;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
public class DynamicOperationType implements OperationType {
private String id;
private String name;
}

View File

@@ -0,0 +1,53 @@
package org.jetlinks.community;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.jetlinks.core.utils.SerializeUtils;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
/**
* 描述,用于对某些操作的通用描述.
*
* @author zhouhao
* @since 2.0
*/
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class Operation implements Externalizable {
/**
* 操作来源
*/
private OperationSource source;
/**
* 操作类型,比如: transparent-codec等
*/
private OperationType type;
@Override
public String toString() {
return type.getId() + "(" + type.getName() + "):[" + source.getId() + "]";
}
@Override
public void writeExternal(ObjectOutput out) throws IOException {
source.writeExternal(out);
SerializeUtils.writeObject(type.getId(), out);
SerializeUtils.writeObject(type.getName(), out);
}
@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
source = new OperationSource();
source.readExternal(in);
type = new DynamicOperationType((String) SerializeUtils.readObject(in), (String) SerializeUtils.readObject(in));
}
}

View File

@@ -0,0 +1,11 @@
package org.jetlinks.community;
public interface OperationType {
String getId();
String getName();
static OperationType of(String id,String name){
return new DynamicOperationType(id,name);
}
}

View File

@@ -1,10 +1,13 @@
package org.jetlinks.community;
import lombok.Generated;
import org.hswebframework.web.id.IDGenerator;
import org.jetlinks.core.config.ConfigKey;
import org.jetlinks.core.message.HeaderKey;
import org.springframework.core.ResolvableType;
import java.lang.reflect.Type;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.function.Supplier;
@@ -13,32 +16,82 @@ import java.util.function.Supplier;
* @author wangzheng
* @since 1.0
*/
@Generated
public interface PropertyConstants {
//机构ID
Key<String> orgId = Key.of("orgId");
//设备名称
Key<String> deviceName = Key.of("deviceName");
//产品名称
Key<String> productName = Key.of("productName");
//产品ID
Key<String> productId = Key.of("productId");
Key<String> uid = Key.of("_uid");
//设备创建者
Key<String> creatorId = Key.of("creatorId");
/**
* 关系信息.值格式:
* <pre>{@code
* [{"type":"user","id":"userId","rel":"manager"}]
* }</pre>
*/
Key<List<Map<String, Object>>> relations = Key.of("relations");
/**
* 租户ID
*
* @see org.jetlinks.pro.tenant.TenantMember
*/
Key<List<String>> tenantId = Key.of("tenantId");
//分组ID
Key<List<String>> groupId = Key.of("groupId");
//是否记录task记录
Key<Boolean> useTask = Key.of("useTask", false);
//taskId
Key<String> taskId = Key.of("taskId");
//最大重试次数
Key<Long> maxRetryTimes = Key.of("maxRetryTimes", () -> Long.getLong("device.message.task.retryTimes", 1), Long.class);
//当前重试次数
Key<Long> retryTimes = Key.of("retryTimes", () -> 0L, Long.class);
//服务ID
Key<String> serverId = Key.of("serverId");
//全局唯一ID
Key<String> uid = Key.of("_uid", IDGenerator.RANDOM::generate);
//设备接入网关ID
Key<String> accessId = Key.of("accessId");
/**
* 设备接入方式
* @see org.jetlinks.community.gateway.supports.DeviceGatewayProvider#getId
*
* @see org.jetlinks.pro.gateway.supports.DeviceGatewayProvider#getId
*/
Key<String> accessProvider = Key.of("accessProvider");
//设备创建者
Key<String> creatorId = Key.of("creatorId");
@SuppressWarnings("all")
static <T> Optional<T> getFromMap(ConfigKey<T> key, Map<String, Object> map) {
return Optional.ofNullable((T) map.get(key.getKey()));
}
@SuppressWarnings("all")
static <T> T getFromMapOrElse(ConfigKey<T> key, Map<String, Object> map, Supplier<T> defaultIfEmpty) {
Object value = map.get(key.getKey());
if (value == null) {
return defaultIfEmpty.get();
}
return (T) value;
}
@Generated
interface Key<V> extends ConfigKey<V>, HeaderKey<V> {
@@ -89,13 +142,14 @@ public interface PropertyConstants {
@Override
public T getDefaultValue() {
return defaultValue.get();
return defaultValue == null ? null : defaultValue.get();
}
};
}
static <T> Key<T> of(String key, Supplier<T> defaultValue, Type type) {
return new Key<T>() {
@Override
public Type getValueType() {
return type;
@@ -108,10 +162,11 @@ public interface PropertyConstants {
@Override
public T getDefaultValue() {
return defaultValue.get();
return defaultValue == null ? null : defaultValue.get();
}
};
}
}
}

View File

@@ -15,7 +15,10 @@ import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.apache.commons.collections4.CollectionUtils;
import org.hswebframework.ezorm.core.param.Term;
import org.hswebframework.web.exception.ValidationException;
import org.hswebframework.web.i18n.LocaleUtils;
import org.jetlinks.community.terms.I18nSpec;
import org.reactivestreams.Subscription;
import org.springframework.util.Assert;
import reactor.core.CoreSubscriber;
@@ -35,6 +38,7 @@ import java.util.*;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;
@Getter
@Setter
@@ -45,6 +49,9 @@ public class TimerSpec implements Serializable {
@NotNull
private Trigger trigger;
@Schema(description = "使用日程标签进行触发")
private Set<String> scheduleTags;
//Cron表达式
@Schema(description = "触发方式为[cron]时不能为空")
private String cron;
@@ -58,9 +65,15 @@ public class TimerSpec implements Serializable {
@Schema(description = "执行模式为[period]时不能为空")
private Period period;
@Schema(description = "执行模式为[period]时不能与period同时为空")
private List<Period> periods;
@Schema(description = "执行模式为[once]时不能为空")
private Once once;
@Schema(description = "组合触发配置列表")
private Multi multi;
public static TimerSpec cron(String cron) {
TimerSpec spec = new TimerSpec();
spec.cron = cron;
@@ -68,6 +81,17 @@ public class TimerSpec implements Serializable {
return spec;
}
public List<Period> periods() {
List<Period> list = new ArrayList<>(1);
if (periods != null) {
list.addAll(periods);
}
if (period != null) {
list.add(period);
}
return list;
}
public Predicate<LocalDateTime> createRangeFilter() {
if (CollectionUtils.isEmpty(when)) {
return ignore -> true;
@@ -82,14 +106,25 @@ public class TimerSpec implements Serializable {
public Predicate<LocalDateTime> createTimeFilter() {
Predicate<LocalDateTime> range = createRangeFilter();
//周期执行指定了to,表示只在时间范围段内执行
if (mod == ExecuteMod.period) {
LocalTime to = period.toLocalTime();
LocalTime from = period.fromLocalTime();
Predicate<LocalDateTime> predicate
= time -> time.toLocalTime().compareTo(from) >= 0;
if (to != null) {
predicate = predicate.and(time -> time.toLocalTime().compareTo(to) <= 0);
Predicate<LocalDateTime> predicate = null;
//可能多个周期
for (Period period : periods()) {
//周期执行指定了to,表示只在时间范围段内执行
LocalTime to = period.toLocalTime();
LocalTime from = period.fromLocalTime();
Predicate<LocalDateTime> _predicate = time -> !time.toLocalTime().isBefore(from);
if (to != null) {
_predicate = _predicate.and(time -> !time.toLocalTime().isAfter(to));
}
if (predicate == null) {
predicate = _predicate;
} else {
predicate = _predicate.or(predicate);
}
}
if (predicate == null) {
return range;
}
return predicate.and(range);
}
@@ -232,6 +267,13 @@ public class TimerSpec implements Serializable {
exception.addSuppressed(e);
throw exception;
}
} else if (trigger == Trigger.multi) {
List<TimerSpec> multiSpec = multi.getSpec();
if (CollectionUtils.isNotEmpty(multiSpec)) {
for (TimerSpec spec : multiSpec) {
spec.validate();
}
}
} else {
nextDurationBuilder().apply(ZonedDateTime.now());
}
@@ -280,6 +322,21 @@ public class TimerSpec implements Serializable {
}
@Getter
@Setter
public static class Multi implements Serializable {
private static final long serialVersionUID = 1L;
@Schema(description = "组合触发配置列表")
private List<TimerSpec> spec;
@Schema(description = "多个触发的关系。and、or")
private Term.Type type = Term.Type.or;
@Schema(description = "组合触发时,只触发一次的最小时间间隔")
private int timeSpanSecond = 2;
}
private static LocalTime parsTime(String time) {
return LocalTime.parse(time);
}
@@ -370,11 +427,40 @@ public class TimerSpec implements Serializable {
}
private TimerIterable periodIterable() {
Assert.notNull(period, "period can not be null");
List<Period> periods = periods();
Assert.notEmpty(periods, "period or periods can not be null");
Predicate<LocalDateTime> filter = createTimeFilter();
Duration duration = Duration.of(period.every, period.unit.temporal);
LocalTime time = period.fromLocalTime();
Duration _Duration = null;
LocalTime earliestFrom = LocalTime.MAX;
LocalTime latestTo = LocalTime.MIN;
//使用最小的执行周期进行判断?
for (Period period : periods) {
Duration duration = Duration.of(period.every, period.unit.temporal);
LocalTime from = period.fromLocalTime();
LocalTime to = period.toLocalTime();
// 更新最小的duration
if (_Duration == null || duration.compareTo(_Duration) < 0) {
_Duration = duration;
}
// 更新最早的起始时间
if (from != null && from.isBefore(earliestFrom)) {
earliestFrom = from;
}
// 更新最晚的结束时间
if (to != null && to.isAfter(latestTo)) {
latestTo = to;
}
}
Duration duration = _Duration;
LocalTime firstFrom = earliestFrom.equals(LocalTime.MAX) ? LocalTime.MIDNIGHT : earliestFrom;
LocalTime endTo = latestTo.equals(LocalTime.MIN) ? null : latestTo;
return baseTime -> new Iterator<ZonedDateTime>() {
ZonedDateTime current = baseTime;
@@ -389,6 +475,21 @@ public class TimerSpec implements Serializable {
int max = MAX_IT_TIMES;
do {
dateTime = dateTime.plus(duration);
// 检查时间是否在 firstFrom 和 endTo 之间
LocalTime time = dateTime.toLocalTime();
if (time.isBefore(firstFrom) || time.isAfter(endTo)) {
// 获取第二天的 firstFrom
ZonedDateTime nextDayFromTime = dateTime.toLocalDate().plusDays(1).atTime(firstFrom).atZone(dateTime.getZone());
// 计算当前时间到 nextDayFromTime 的差值
Duration timeDifference = Duration.between(dateTime, nextDayFromTime);
// 计算可以整除的 duration 数量
long n = timeDifference.toMillis() / duration.toMillis();
// 跳转到下一个 n * duration 的时间点
dateTime = dateTime.plus(n * duration.toMillis(), ChronoUnit.MILLIS);
}
if (filter.test(dateTime.toLocalDateTime())) {
break;
}
@@ -432,7 +533,110 @@ public class TimerSpec implements Serializable {
};
}
private TimerIterable multiSpecIterable() {
List<TimerSpec> multiSpec = multi.getSpec();
Assert.notEmpty(multiSpec, "multiSpec can not be empty");
return baseTime -> new Iterator<ZonedDateTime>() {
final List<ZonedDateTime> timeList = new ArrayList<>(multiSpec.size());
final List<Iterator<ZonedDateTime>> iterators = multiSpec
.stream()
.map(spec -> spec.iterable().iterator(baseTime))
.collect(Collectors.toList());
@Override
public boolean hasNext() {
switch (multi.getType()) {
case and:
return iterators.stream().allMatch(Iterator::hasNext);
case or:
return iterators.stream().anyMatch(Iterator::hasNext);
default:
return false;
}
}
@Override
public ZonedDateTime next() {
switch (multi.getType()) {
case and:
return handleNextAnd();
case or:
return handleNextOr();
default:
return baseTime;
}
}
private ZonedDateTime handleNextAnd() {
ZonedDateTime dateTime = null;
int max = MAX_IT_TIMES;
int match = 0;
do {
for (Iterator<ZonedDateTime> iterator : iterators) {
ZonedDateTime next = iterator.next();
if (dateTime == null) {
dateTime = next;
}
// 若生成的时间比当前选中的时间早,则继续生成
while (next.isBefore(dateTime)) {
next = iterator.next();
}
if (next.isEqual(dateTime)) {
// 所有定时器的next时间一致时返回时间
if (++match == iterators.size()) {
return dateTime;
}
} else {
dateTime = next;
}
}
max--;
} while (
max > 0
);
return dateTime;
}
private ZonedDateTime handleNextOr() {
ZonedDateTime earliest = null;
// 每个定时器生成next
fillTimeList();
// 获取最早的一个时间
for (ZonedDateTime zonedDateTime : timeList) {
if (earliest == null || earliest.isAfter(zonedDateTime) || earliest.isEqual(zonedDateTime)) {
earliest = zonedDateTime;
}
}
// 清空被选中的最早时间
for (int i = 0; i < timeList.size(); i++) {
if (timeList.get(i).isEqual(earliest)) {
timeList.set(i, null);
}
}
return earliest;
}
/**
* 遍历所有定时器若有next为空的则生成新的
*/
private void fillTimeList() {
for (int i = 0; i < iterators.size(); i++) {
if (timeList.size() <= i) {
timeList.add(iterators.get(i).next());
} else if (timeList.get(i) == null) {
timeList.set(i, iterators.get(i).next());
}
}
}
};
}
public TimerIterable iterable() {
if (trigger == Trigger.multi) {
return multiSpecIterable();
}
if ((trigger == Trigger.cron || trigger == null) && cron != null) {
return cronIterable();
}
@@ -448,7 +652,6 @@ public class TimerSpec implements Serializable {
return timeList;
}
public Flux<Long> flux() {
return flux(Schedulers.parallel());
}
@@ -457,9 +660,111 @@ public class TimerSpec implements Serializable {
return new TimerFlux(nextDurationBuilder(), scheduler);
}
@Override
public String toString() {
if (getTrigger() == null) {
return null;
}
switch (getTrigger()) {
case week: {
return weekDesc();
}
case month: {
return monthDesc();
}
case cron: {
return getCron();
}
}
return null;
}
private String weekDesc() {
I18nSpec spec = new I18nSpec();
spec.setCode("message.timer_spec_desc");
List<I18nSpec> args = new ArrayList<>();
if (when == null || when.isEmpty()) {
args.add(I18nSpec.of("message.timer_spec_desc_everyday", "每天"));
} else {
String week = when
.stream()
.map(weekNum -> LocaleUtils.resolveMessage("message.timer_spec_desc_week_" + weekNum))
.collect(Collectors.joining(LocaleUtils.resolveMessage("message.timer_spec_desc_seperator")));
args.add(I18nSpec.of(
"message.timer_spec_desc_everyweek",
"每周" + week,
week));
}
args.add(timerModDesc());
spec.setArgs(args);
return spec.resolveI18nMessage();
}
private String monthDesc() {
I18nSpec spec = new I18nSpec();
spec.setCode("message.timer_spec_desc");
List<I18nSpec> args = new ArrayList<>();
if (when == null || when.isEmpty()) {
args.add(I18nSpec.of("message.timer_spec_desc_everyday", "每天"));
} else {
String month = when
.stream()
.map(monthNum -> {
switch (monthNum) {
case 1:
return LocaleUtils.resolveMessage("message.timer_spec_desc_month_1", monthNum);
case 2:
return LocaleUtils.resolveMessage("message.timer_spec_desc_month_2", monthNum);
case 3:
return LocaleUtils.resolveMessage("message.timer_spec_desc_month_3", monthNum);
default:
return LocaleUtils.resolveMessage("message.timer_spec_desc_month", monthNum);
}
})
.collect(Collectors.joining(LocaleUtils.resolveMessage("message.timer_spec_desc_seperator")));
args.add(I18nSpec.of(
"message.timer_spec_desc_everymonth",
"每月" + month,
month));
}
args.add(timerModDesc());
spec.setArgs(args);
return spec.resolveI18nMessage();
}
private I18nSpec timerModDesc() {
switch (getMod()) {
case period: {
if (getPeriod() == null) {
break;
}
return I18nSpec.of(
"message.timer_spec_desc_period",
getPeriod().getFrom() + "-" + getPeriod().getTo() +
"" + getPeriod().getEvery() + getPeriod().getUnit().name(),
I18nSpec.of("message.timer_spec_desc_period_duration",
getPeriod().getFrom() + "-" + getPeriod().getTo(),
getPeriod().getFrom(),
getPeriod().getTo()),
getPeriod().getEvery(),
I18nSpec.of("message.timer_spec_desc_period_" + getPeriod().getUnit().name(),
getPeriod().getUnit().name())
);
}
case once: {
// [time]执行1次
if (getOnce() == null) {
break;
}
return I18nSpec.of("message.timer_spec_desc_period_once", getOnce().getTime(), getOnce().getTime());
}
}
return I18nSpec.of("", "");
}
@AllArgsConstructor
static class TimerFlux extends Flux<Long> {
final Function<ZonedDateTime, Duration> spec;
final Function<ZonedDateTime, Duration> spec;
final Scheduler scheduler;
@Override
@@ -530,9 +835,14 @@ public class TimerSpec implements Serializable {
}
public enum Trigger {
//按周
week,
//按月
month,
cron
//cron表达式
cron,
// 多个触发组合
multi
}
public enum ExecuteMod {

View File

@@ -0,0 +1,64 @@
package org.jetlinks.community.annotation.command;
import org.jetlinks.core.annotation.command.CommandHandler;
import org.springframework.stereotype.Indexed;
import java.lang.annotation.*;
/**
* 标记一个类为命令服务支持端点,用于对外提供命令支持
* <pre>{@code
*
* @CommandService("myService")
* public class MyCommandService{
*
* }
*
* }</pre>
*
* @author zhouhao
* @since 1.2.3
*/
@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Inherited
@Indexed
public @interface CommandService {
/**
* 服务标识
*
* @return 服务标识
*/
String id();
/**
* 服务名称
*
* @return 服务名称
*/
String name();
/**
* 服务描述
*
* @return 服务描述
*/
String[] description() default {};
/**
* 是否根据注解扫描注册服务
*
* @return 是否注册服务
*/
boolean autoRegistered() default true;
/**
* 命令定义,用于声明支持的命令
*
* @return 命令定义
*/
CommandHandler[] commands() default {};
}

View File

@@ -0,0 +1,433 @@
package org.jetlinks.community.authorize;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.google.common.collect.Collections2;
import com.google.common.collect.Maps;
import com.google.common.collect.Sets;
import lombok.AllArgsConstructor;
import lombok.Setter;
import lombok.SneakyThrows;
import org.apache.commons.lang3.StringUtils;
import org.hswebframework.web.authorization.*;
import org.hswebframework.web.authorization.simple.*;
import org.hswebframework.web.bean.FastBeanCopier;
import org.jetlinks.core.metadata.Jsonable;
import org.jetlinks.core.utils.RecyclerUtils;
import org.jetlinks.core.utils.SerializeUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.util.*;
import java.util.function.Function;
import java.util.stream.Collectors;
@Setter
public class FastSerializableAuthentication extends SimpleAuthentication
implements Externalizable, Jsonable {
private static final Logger log = LoggerFactory.getLogger(FastSerializableAuthentication.class);
static {
SerializeUtils.registerSerializer(
0x90,
FastSerializableAuthentication.class,
(ignore) -> new FastSerializableAuthentication());
}
public static void load() {
}
@SuppressWarnings("all")
public static Authentication of(Object jsonOrObject, boolean share) {
if (jsonOrObject == null) {
return null;
}
//json
if (jsonOrObject instanceof String) {
return of((String) jsonOrObject, share);
}
// map
if (jsonOrObject instanceof Map) {
FastSerializableAuthentication fast = new FastSerializableAuthentication();
fast.shared = share;
fast.fromJson(new JSONObject((Map) jsonOrObject));
return fast;
}
//auth
if (jsonOrObject instanceof Authentication) {
return of(((Authentication) jsonOrObject));
}
throw new UnsupportedOperationException();
}
public static Authentication of(String json, boolean share) {
if (StringUtils.isEmpty(json)) {
return null;
}
FastSerializableAuthentication fast = new FastSerializableAuthentication();
fast.shared = share;
fast.fromJson(JSON.parseObject(json));
return fast;
}
public static Authentication of(Authentication auth) {
return of(auth, false);
}
public static Authentication of(Authentication auth, boolean simplify) {
if (auth instanceof FastSerializableAuthentication) {
((FastSerializableAuthentication) auth).simplify = simplify;
return auth;
}
FastSerializableAuthentication fast = new FastSerializableAuthentication();
fast.setUser(auth.getUser());
fast.setSimplify(simplify);
fast.getPermissions().addAll(auth.getPermissions());
fast.getDimensions().addAll(auth.getDimensions());
fast.getAttributes().putAll(auth.getAttributes());
return fast;
}
@Override
protected FastSerializableAuthentication newInstance() {
return new FastSerializableAuthentication();
}
/**
* 是否简化,为true时,不序列化权限名称
*
* @see Permission#getName()
*/
private boolean simplify = false;
private transient boolean shared;
public void makeShared() {
shared = true;
List<Dimension> dimensions = getDimensions()
.stream()
.map(RecyclerUtils::intern)
.collect(Collectors.toList());
setDimensions(dimensions);
}
@Override
public void writeExternal(ObjectOutput out) throws IOException {
String userId = null;
try {
out.writeByte(0x01);
//是否简化模式
out.writeBoolean(simplify);
//user
User user = getUser();
out.writeUTF(user.getId() == null ? "" : (userId = user.getId()));
SerializeUtils.writeNullableUTF(user.getName(), out);
out.writeUTF(user.getUsername() == null ? "" : user.getUsername());
SerializeUtils.writeNullableUTF(user.getUserType(), out);
SerializeUtils.writeKeyValue(user.getOptions(), out);
//permission
{
List<Permission> permissions = getPermissions();
if (permissions == null) {
permissions = Collections.emptyList();
}
out.writeInt(permissions.size());
for (Permission permission : permissions) {
write(permission, out);
}
}
//dimension
{
List<Dimension> dimensions = getDimensions();
if (dimensions == null) {
dimensions = Collections.emptyList();
}
out.writeInt(dimensions.size());
for (Dimension permission : dimensions) {
write(permission, out);
}
}
SerializeUtils.writeKeyValue(getAttributes(), out);
} catch (Throwable e) {
log.warn("write FastSerializableAuthentication [{}] error", userId, e);
throw e;
}
}
@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
byte version = in.readByte();
simplify = in.readBoolean();
//user
SimpleUser user = new SimpleUser();
user.setId(in.readUTF());
user.setName(SerializeUtils.readNullableUTF(in));
user.setUsername(in.readUTF());
user.setUserType(SerializeUtils.readNullableUTF(in));
user.setOptions(SerializeUtils.readMap(in, Maps::newHashMapWithExpectedSize));
setUser0(user);
//permission
{
int size = in.readInt();
List<Permission> permissions = new ArrayList<>(size);
for (int i = 0; i < size; i++) {
Permission permission = readPermission(in);
permissions.add(permission);
}
setPermissions(permissions);
}
//dimension
{
int size = in.readInt();
Set<Dimension> dimensions = new HashSet<>(size);
for (int i = 0; i < size; i++) {
Dimension dimension = readDimension(in);
dimensions.add(dimension);
}
setDimensions(dimensions);
}
setAttributes(SerializeUtils.readMap(in, Maps::newHashMapWithExpectedSize));
}
@SneakyThrows
private Dimension readDimension(ObjectInput in) {
SimpleDimension dimension = new SimpleDimension();
dimension.setId(in.readUTF());
dimension.setName(in.readUTF());
dimension.setOptions(SerializeUtils.readMap(
in,
k -> RecyclerUtils.intern(String.valueOf(k)),
Function.identity(),
Maps::newHashMapWithExpectedSize));
boolean known = in.readBoolean();
if (known) {
KnownDimension knownDimension = KnownDimension.ALL[in.readByte()];
dimension.setType(knownDimension.type);
} else {
SimpleDimensionType type = new SimpleDimensionType();
type.setId(in.readUTF());
type.setName(in.readUTF());
dimension.setType(RecyclerUtils.intern(type));
}
return dimension;
}
@SneakyThrows
private void write(Dimension dimension, ObjectOutput out) {
out.writeUTF(dimension.getId());
out.writeUTF(dimension.getName() == null ? "" : dimension.getName());
SerializeUtils.writeKeyValue(dimension.getOptions(), out);
KnownDimension knownDimension = KnownDimension.MAPPING.get(dimension.getType().getId());
out.writeBoolean(knownDimension != null);
if (knownDimension != null) {
out.writeByte(knownDimension.ordinal());
} else {
out.writeUTF(dimension.getType().getId());
out.writeUTF(dimension.getType().getName());
}
}
@SneakyThrows
private Permission readPermission(ObjectInput in) {
SimplePermission permission = new SimplePermission();
permission.setId(in.readUTF());
if (!simplify) {
permission.setName(in.readUTF());
} else {
permission.setName(permission.getId());
}
permission.setOptions(SerializeUtils.readMap(in, Maps::newHashMapWithExpectedSize));
int actionSize = in.readUnsignedShort();
Set<String> actions = Sets.newHashSetWithExpectedSize(actionSize);
for (int i = 0; i < actionSize; i++) {
if (in.readBoolean()) {
actions.add(KnownAction.ALL[in.readByte()].action);
} else {
actions.add(in.readUTF());
}
}
permission.setActions(actions);
return permission;
}
@SneakyThrows
private void write(Permission permission, ObjectOutput out) {
out.writeUTF(permission.getId());
if (!simplify) {
out.writeUTF(permission.getName() == null ? "" : permission.getName());
}
SerializeUtils.writeKeyValue(permission.getOptions(), out);
Set<String> actions = permission.getActions();
out.writeShort(actions.size());
for (String action : actions) {
KnownAction knownAction = KnownAction.ACTION_MAP.get(action);
out.writeBoolean(knownAction != null);
if (null != knownAction) {
out.writeByte(knownAction.ordinal());
} else {
out.writeUTF(action);
}
}
}
@AllArgsConstructor
enum KnownDimension {
user(DefaultDimensionType.user),
role(DefaultDimensionType.role),
org(OrgDimensionType.org),
parentOrg(OrgDimensionType.parentOrg);
private final DimensionType type;
static final KnownDimension[] ALL = values();
static final Map<Object, KnownDimension> MAPPING = new HashMap<>();
static {
for (KnownDimension value : ALL) {
MAPPING.put(value, value);
MAPPING.put(value.ordinal(), value);
MAPPING.put(value.name(), value);
}
}
}
enum KnownAction {
query,
get,
update,
save,
delete,
export,
_import(Permission.ACTION_IMPORT),
enable,
disable;
static final KnownAction[] ALL = values();
static final Map<Object, KnownAction> ACTION_MAP = new HashMap<>();
static {
for (KnownAction value : ALL) {
ACTION_MAP.put(value, value);
ACTION_MAP.put(value.ordinal(), value);
ACTION_MAP.put(value.action, value);
}
}
private final String action;
KnownAction() {
this.action = name();
}
KnownAction(String action) {
this.action = action;
}
}
@Override
public JSONObject toJson() {
JSONObject obj = new JSONObject();
obj.put("user", SerializeUtils.convertToSafelySerializable(getUser()));
obj.put("permissions", SerializeUtils.convertToSafelySerializable(getPermissions()));
//忽略user
obj.put("dimensions", SerializeUtils.convertToSafelySerializable(
Collections2.filter(getDimensions(), i -> !(i instanceof User))
));
obj.put("attributes", new HashMap<>(getAttributes()));
return obj;
}
@Override
public void fromJson(JSONObject json) {
JSONObject user = json.getJSONObject("user");
if (user != null) {
setUser(user.toJavaObject(SimpleUser.class));
}
JSONArray permissions = json.getJSONArray("permissions");
if (permissions != null) {
for (int i = 0, size = permissions.size(); i < size; i++) {
JSONObject permission = permissions.getJSONObject(i);
//不再支持
permission.remove("dataAccesses");
Object actions = permission.remove("actions");
SimplePermission perm = permission.toJavaObject(SimplePermission.class);
if (actions instanceof Collection) {
@SuppressWarnings("all")
Collection<Object> _actions = (Collection<Object>) actions;
Set<String> acts = Sets.newHashSetWithExpectedSize(_actions.size());
for (Object action : _actions) {
KnownAction act = KnownAction.ACTION_MAP.get(action);
if (act == null) {
acts.add(String.valueOf(action));
} else {
acts.add(act.action);
}
}
perm.setActions(acts);
}
getPermissions().add(shared ? RecyclerUtils.intern(perm) : perm);
}
}
JSONArray dimensions = json.getJSONArray("dimensions");
if (dimensions != null) {
for (int i = 0, size = dimensions.size(); i < size; i++) {
JSONObject dimension = dimensions.getJSONObject(i);
Object type = dimension.remove("type");
if (type == null) {
continue;
}
SimpleDimension simpleDimension = dimension.toJavaObject(SimpleDimension.class);
if (type instanceof DimensionType) {
simpleDimension.setType((DimensionType) type);
} else {
KnownDimension knownDimension = KnownDimension.MAPPING.get(type);
if (knownDimension != null) {
simpleDimension.setType(knownDimension.type);
} else {
SimpleDimensionType dimensionType;
if (type instanceof String) {
dimensionType = SimpleDimensionType.of(String.valueOf(type));
} else {
dimensionType = FastBeanCopier.copy(type, new SimpleDimensionType());
}
if (StringUtils.isNoneEmpty(dimensionType.getId())) {
simpleDimension.setType(shared ? RecyclerUtils.intern(dimensionType) : dimensionType);
}
}
}
getDimensions().add(shared ? RecyclerUtils.intern(simpleDimension) : simpleDimension);
}
}
JSONObject attr = json.getJSONObject("attributes");
if (attr != null) {
getAttributes().putAll(Maps.transformValues(attr, Serializable.class::cast));
}
}
}

View File

@@ -0,0 +1,22 @@
package org.jetlinks.community.authorize;
import lombok.AllArgsConstructor;
import lombok.Generated;
import lombok.Getter;
import org.hswebframework.web.authorization.DimensionType;
/**
* @author wangzheng
* @since 1.0
*/
@AllArgsConstructor
@Getter
@Generated
public enum OrgDimensionType implements DimensionType {
org("org","组织"),
parentOrg("parentOrg","上级组织");
private final String id;
private final String name;
}

View File

@@ -0,0 +1,81 @@
package org.jetlinks.community.buffer;
import org.jetlinks.community.Operation;
import org.jetlinks.community.OperationSource;
import org.jetlinks.community.OperationType;
import org.jetlinks.community.event.SystemEventHolder;
import org.jetlinks.community.utils.TimeUtils;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicIntegerFieldUpdater;
import java.util.concurrent.atomic.AtomicLongFieldUpdater;
public abstract class AbstractBufferEviction implements BufferEviction {
public static final OperationType OPERATION_TYPE = OperationType.of("buffer-eviction", "缓冲区数据丢弃");
private static final AtomicLongFieldUpdater<AbstractBufferEviction>
LAST_EVENT_TIME = AtomicLongFieldUpdater.newUpdater(AbstractBufferEviction.class, "lastEventTime");
private static final AtomicIntegerFieldUpdater<AbstractBufferEviction>
LAST_TIMES = AtomicIntegerFieldUpdater.newUpdater(AbstractBufferEviction.class, "lastTimes");
//最大事件推送频率
//可通过java -Djetlinks.buffer.eviction.event.max-interval=10m修改配置
private static final long MAX_EVENT_INTERVAL =
TimeUtils.parse(System.getProperty("jetlinks.buffer.eviction.event.max-interval", "10m")).toMillis();
private volatile long lastEventTime;
private volatile int lastTimes;
abstract boolean doEviction(EvictionContext context);
@Override
public boolean tryEviction(EvictionContext context) {
if (doEviction(context)) {
sendEvent(context);
return true;
}
return false;
}
private String operationCode() {
return getClass().getSimpleName();
}
private void sendEvent(EvictionContext context) {
long now = System.currentTimeMillis();
long time = LAST_EVENT_TIME.get(this);
//记录事件推送周期内总共触发了多少次
LAST_TIMES.incrementAndGet(this);
//超过间隔事件则推送事件,防止推送太多错误事件
if (now - time > MAX_EVENT_INTERVAL) {
LAST_EVENT_TIME.set(this, now);
Map<String, Object> info = new HashMap<>();
//缓冲区数量
info.put("bufferSize", context.size(EvictionContext.BufferType.buffer));
//死数据数量
info.put("deadSize", context.size(EvictionContext.BufferType.dead));
//总计触发次数
info.put("times", LAST_TIMES.getAndSet(this, 0));
//应用自定义的数据,比如磁盘剩余空间等信息
applyEventData(info);
//推送系统事件
SystemEventHolder.warn(
Operation.of(
OperationSource.of(context.getName(), "eviction"),
OPERATION_TYPE),
operationCode(),
info
);
}
}
protected void applyEventData(Map<String, Object> data) {
}
}

View File

@@ -0,0 +1,133 @@
package org.jetlinks.community.buffer;
import org.springframework.util.unit.DataSize;
import java.io.File;
/**
* 缓存淘汰策略
*
* @author zhouhao
* @since 2.0
*/
public interface BufferEviction {
BufferEviction NONE = new BufferEviction() {
@Override
public boolean tryEviction(EvictionContext context) {
return false;
}
@Override
public String toString() {
return "None";
}
};
/**
* 根据磁盘使用率来进行淘汰,磁盘使用率超过阈值时则淘汰旧数据
*
* @param path 文件路径
* @param threshold 使用率阈值 范围为0-1 .如: 0.8 表示磁盘使用率超过80%则丢弃数据
* @return 淘汰策略
*/
static BufferEviction disk(String path, float threshold) {
return new DiskUsageEviction(new File(path), threshold);
}
/**
* 根据磁盘可用空间来进行淘汰,磁盘剩余空间低于阈值时则淘汰旧数据
*
* @param path 文件路径
* @param minUsableDataSize 磁盘最小可用空间阈值,当磁盘可用空间低于此值时则则淘汰旧数据
* @return 淘汰策略
*/
static BufferEviction disk(String path, DataSize minUsableDataSize) {
return new DiskFreeEviction(new File(path), minUsableDataSize.toBytes());
}
/**
* 根据缓冲区数量来淘汰数据,当数量超过指定阈值后则淘汰旧数据
*
* @param bufferLimit 数量阈值
* @return 淘汰策略
*/
static BufferEviction limit(long bufferLimit) {
return limit(bufferLimit, bufferLimit);
}
/**
* 根据缓冲区数量来淘汰数据,当数量超过指定阈值后则淘汰旧数据
*
* @param bufferLimit 缓冲数量阈值
* @param deadLimit 死数据数量阈值
* @return 淘汰策略
*/
static BufferEviction limit(long bufferLimit, long deadLimit) {
return new SizeLimitEviction(bufferLimit, deadLimit);
}
/**
* 根据缓冲区数量来淘汰死数据
*
* @param deadLimit 死数据数量阈值
* @return 淘汰策略
*/
static BufferEviction deadLimit(long deadLimit) {
return new SizeLimitEviction(-1, deadLimit);
}
/**
* 尝试执行淘汰
*
* @param context 上下文
* @return 是否有数据被淘汰
*/
boolean tryEviction(EvictionContext context);
/**
* 组合另外一个淘汰策略,2个策略同时执行.
*
* @param after 后续策略
* @return 淘汰策略
*/
default BufferEviction and(BufferEviction after) {
BufferEviction self = this;
return new BufferEviction() {
@Override
public boolean tryEviction(EvictionContext context) {
return self.tryEviction(context) & after.tryEviction(context);
}
@Override
public String toString() {
return self + " and " + after;
}
};
}
/**
* 组合另外一个淘汰策略,当前策略淘汰了数据才执行另外一个策略
*
* @param after 后续策略
* @return 淘汰策略
*/
default BufferEviction then(BufferEviction after) {
BufferEviction self = this;
return new BufferEviction() {
@Override
public boolean tryEviction(EvictionContext context) {
if (self.tryEviction(context)) {
after.tryEviction(context);
return true;
}
return false;
}
@Override
public String toString() {
return self + " then " + after;
}
};
}
}

View File

@@ -0,0 +1,55 @@
package org.jetlinks.community.buffer;
import lombok.Getter;
import lombok.Setter;
import org.springframework.util.unit.DataSize;
@Getter
@Setter
public class BufferEvictionSpec {
public static final BufferEviction DEFAULT = new BufferEvictionSpec().build();
//最大队列数量,超过则淘汰最旧的数据
private int maxSize = -1;
//最大死信数量,超过则淘汰dead数据
private int maxDeadSize = Integer.getInteger("jetlinks.buffer.dead.limit", 100_0000);
//根据磁盘空间淘汰数据
private DataSize diskFree = DataSize.parse(System.getProperty("jetlinks.buffer.disk.free.threshold", "4GB"));
//磁盘最大使用率
private float diskThreshold;
//判断磁盘空间大小的目录
private String diskPath = System.getProperty("jetlinks.buffer.disk.free.path", "./");
public BufferEviction build() {
BufferEviction
eviction = null,
size = BufferEviction.limit(maxSize, maxDeadSize),
disk = null;
if (diskThreshold > 0) {
disk = BufferEviction.disk(diskPath, diskThreshold);
} else if (diskFree != null) {
disk = BufferEviction.disk(diskPath, diskFree);
}
if (disk != null) {
eviction = disk;
}
if (eviction == null) {
eviction = size;
} else {
eviction = eviction.then(size);
}
return eviction;
}
}

View File

@@ -23,6 +23,15 @@ public class BufferProperties {
//最大重试次数,超过此次数的数据将会放入死队列.
private long maxRetryTimes = 64;
//文件操作的最大并行度,默认为1,不建议设置超过4.
private int fileConcurrency = 1;
//消费策略 默认先进先出
private ConsumeStrategy strategy = ConsumeStrategy.FIFO;
//淘汰策略
private BufferEvictionSpec eviction = new BufferEvictionSpec();
public boolean isExceededRetryCount(int count) {
return maxRetryTimes > 0 && count >= maxRetryTimes;
}

View File

@@ -10,19 +10,18 @@ import org.springframework.transaction.CannotCreateTransactionException;
import java.io.IOException;
import java.time.Duration;
import java.util.Objects;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.TimeoutException;
import java.util.function.Predicate;
/**
* @author zhouhao
* @since 2.0
*/
@Getter
@AllArgsConstructor
public class BufferSettings {
private static final Predicate<Throwable> DEFAULT_RETRY_WHEN_ERROR =
e -> ErrorUtils.hasException(e, IOException.class,
IllegalStateException.class,
RejectedExecutionException.class,
TimeoutException.class,
DataAccessResourceFailureException.class,
CannotCreateTransactionException.class,
@@ -32,10 +31,17 @@ public class BufferSettings {
return DEFAULT_RETRY_WHEN_ERROR;
}
public static BufferEviction defaultEviction(){
return BufferEvictionSpec.DEFAULT;
}
private final String filePath;
private final String fileName;
//缓存淘汰策略
private final BufferEviction eviction;
private final Predicate<Throwable> retryWhenError;
//缓冲区大小,超过此大小将执行 handler 处理逻辑
@@ -50,16 +56,23 @@ public class BufferSettings {
//最大重试次数,超过此次数的数据将会放入死队列.
private final long maxRetryTimes;
private final int fileConcurrency;
private final ConsumeStrategy strategy;
public static BufferSettings create(String filePath, String fileName) {
return new BufferSettings(
filePath,
fileName,
defaultEviction(),
//默认重试逻辑
defaultRetryWhenError(),
1000,
Duration.ofSeconds(1),
Math.max(1, Runtime.getRuntime().availableProcessors() / 2),
5);
5,
1,
ConsumeStrategy.FIFO);
}
public static BufferSettings create(BufferProperties properties) {
@@ -70,64 +83,122 @@ public class BufferSettings {
return create(properties.getFilePath(), fileName).properties(properties);
}
public BufferSettings bufferSize(int bufferSize) {
public BufferSettings eviction(BufferEviction eviction) {
return new BufferSettings(filePath,
fileName,
eviction,
retryWhenError,
bufferSize,
bufferTimeout,
parallelism,
maxRetryTimes);
maxRetryTimes,
fileConcurrency,
strategy);
}
public BufferSettings bufferSize(int bufferSize) {
return new BufferSettings(filePath,
fileName,
eviction,
retryWhenError,
bufferSize,
bufferTimeout,
parallelism,
maxRetryTimes,
fileConcurrency,
strategy);
}
public BufferSettings bufferTimeout(Duration bufferTimeout) {
return new BufferSettings(filePath,
fileName,
eviction,
retryWhenError,
bufferSize,
bufferTimeout,
parallelism,
maxRetryTimes);
maxRetryTimes,
fileConcurrency,
strategy);
}
public BufferSettings parallelism(int parallelism) {
return new BufferSettings(filePath,
fileName,
eviction,
retryWhenError,
bufferSize,
bufferTimeout,
parallelism,
maxRetryTimes);
maxRetryTimes,
fileConcurrency,
strategy);
}
public BufferSettings maxRetry(int maxRetryTimes) {
return new BufferSettings(filePath,
fileName,
eviction,
retryWhenError,
bufferSize,
bufferTimeout,
parallelism,
maxRetryTimes);
maxRetryTimes,
fileConcurrency,
strategy);
}
public BufferSettings retryWhenError(Predicate<Throwable> retryWhenError) {
return new BufferSettings(filePath,
fileName,
eviction,
Objects.requireNonNull(retryWhenError),
bufferSize,
bufferTimeout,
parallelism,
maxRetryTimes);
maxRetryTimes,
fileConcurrency,
strategy);
}
public BufferSettings fileConcurrency(int fileConcurrency) {
return new BufferSettings(filePath,
fileName,
eviction,
retryWhenError,
bufferSize,
bufferTimeout,
parallelism,
maxRetryTimes,
fileConcurrency,
strategy);
}
public BufferSettings strategy(ConsumeStrategy strategy) {
return new BufferSettings(filePath,
fileName,
eviction,
retryWhenError,
bufferSize,
bufferTimeout,
parallelism,
maxRetryTimes,
fileConcurrency,
strategy);
}
public BufferSettings properties(BufferProperties properties) {
return new BufferSettings(filePath,
fileName,
properties.getEviction().build(),
Objects.requireNonNull(retryWhenError),
properties.getSize(),
properties.getTimeout(),
properties.getParallelism(),
properties.getMaxRetryTimes());
properties.getMaxRetryTimes(),
properties.getFileConcurrency(),
properties.getStrategy()
);
}

View File

@@ -0,0 +1,10 @@
package org.jetlinks.community.buffer;
public enum ConsumeStrategy {
// 先进先出
FIFO,
// 后进先出
LIFO
}

View File

@@ -0,0 +1,59 @@
package org.jetlinks.community.buffer;
import org.jetlinks.community.utils.FormatUtils;
import org.springframework.util.unit.DataSize;
import java.io.File;
import java.util.Map;
class DiskFreeEviction extends AbstractBufferEviction {
private final File path;
private final long minUsableBytes;
public DiskFreeEviction(File path, long minUsableBytes) {
this.path = path;
this.minUsableBytes = minUsableBytes;
}
private volatile long usableSpace = -1;
private volatile long lastUpdateTime;
@Override
public boolean doEviction(EvictionContext context) {
tryUpdate();
if (freeOutOfThreshold()) {
context.removeOldest(EvictionContext.BufferType.buffer);
return true;
}
return false;
}
protected boolean freeOutOfThreshold() {
return usableSpace != -1 && usableSpace <= minUsableBytes;
}
private void tryUpdate() {
long now = System.currentTimeMillis();
//1秒更新一次
if (now - lastUpdateTime <= 1000) {
return;
}
usableSpace = path.getUsableSpace();
lastUpdateTime = now;
}
@Override
protected void applyEventData(Map<String, Object> data) {
data.put("usableSpace", DataSize.ofBytes(usableSpace).toMegabytes());
data.put("minUsableBytes", DataSize.ofBytes(minUsableBytes).toMegabytes());
}
@Override
public String toString() {
return "DiskFree(path=" + path
+ ",space=" + FormatUtils.formatDataSize(usableSpace) + "/" + FormatUtils.formatDataSize(minUsableBytes) + ")";
}
}

View File

@@ -0,0 +1,59 @@
package org.jetlinks.community.buffer;
import java.io.File;
import java.util.Map;
class DiskUsageEviction extends AbstractBufferEviction {
private final File path;
private final float threshold;
public DiskUsageEviction(File path, float threshold) {
this.path = path;
this.threshold = threshold;
}
private volatile float usage;
private volatile long lastUpdateTime;
@Override
public boolean doEviction(EvictionContext context) {
tryUpdate();
if (freeOutOfThreshold()) {
context.removeOldest(EvictionContext.BufferType.buffer);
return true;
}
return false;
}
protected boolean freeOutOfThreshold() {
return usage >= threshold;
}
private void tryUpdate() {
long now = System.currentTimeMillis();
//1秒更新一次
if (now - lastUpdateTime <= 1000) {
return;
}
long total = path.getTotalSpace();
long usable = path.getUsableSpace();
usage = (float) ((total - usable) / (double) total);
lastUpdateTime = now;
}
@Override
protected void applyEventData(Map<String, Object> data) {
data.put("usage", String.format("%.2f%%", usage * 100));
}
@Override
public String toString() {
return "DiskUsage(path=" + path
+ ", threshold=" + String.format("%.2f%%", threshold * 100)
+ ", usage=" + String.format("%.2f%%", usage * 100) + ")";
}
}

View File

@@ -0,0 +1,44 @@
package org.jetlinks.community.buffer;
/**
* 缓冲淘汰上下文
*
* @author zhouhao
* @since 2.0
*/
public interface EvictionContext {
/**
* 获取指定类型的数据量
*
* @param type 类型
* @return 数据量
*/
long size(BufferType type);
/**
* 删除最新的数据
*
* @param type 类型
*/
void removeLatest(BufferType type);
/**
* 删除最旧的数据
*
* @param type 类型
*/
void removeOldest(BufferType type);
/**
* @return 缓冲区名称, 用于区分多个不同的缓冲区
*/
String getName();
enum BufferType {
//缓冲区
buffer,
//死数据
dead
}
}

View File

@@ -5,14 +5,16 @@ import io.netty.buffer.*;
import io.netty.util.ReferenceCountUtil;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import org.apache.commons.lang3.exception.ExceptionUtils;
import org.h2.mvstore.WriteBuffer;
import org.h2.mvstore.type.BasicDataType;
import org.jetlinks.community.codec.Serializers;
import org.jetlinks.core.cache.FileQueue;
import org.jetlinks.core.cache.FileQueueProxy;
import org.jetlinks.core.utils.SerializeUtils;
import org.jetlinks.community.codec.Serializers;
import org.jetlinks.community.utils.FormatUtils;
import org.reactivestreams.Subscription;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -21,9 +23,15 @@ import reactor.core.publisher.BaseSubscriber;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.core.publisher.SignalType;
import reactor.core.scheduler.Scheduler;
import reactor.core.scheduler.Schedulers;
import javax.annotation.Nonnull;
import javax.management.MBeanServer;
import javax.management.ObjectName;
import javax.management.StandardMBean;
import java.io.*;
import java.lang.management.ManagementFactory;
import java.lang.reflect.Array;
import java.nio.ByteBuffer;
import java.nio.file.Path;
@@ -38,6 +46,7 @@ import java.util.function.BiFunction;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.function.Supplier;
import java.util.stream.Collectors;
/**
* 支持持久化的缓存批量操作工具,用于支持数据的批量操作,如批量写入数据到数据库等.
@@ -46,11 +55,13 @@ import java.util.function.Supplier;
*
* <pre>{@code
*
* BufferWriter<Data> writer = BufferWriter
* PersistenceBuffer<Data> writer = PersistenceBuffer
* .<Data>create(
* "./data/buffer", //文件目录
* "my-data.queue", //文件名
* Data::new,
* buffer->{
* // 返回false表示不重试
* return saveData(buffer);
* })
* .bufferSize(1000)//缓冲大小,当缓冲区超过此数量时将会立即执行写出操作.
@@ -66,9 +77,9 @@ import java.util.function.Supplier;
*
* @param <T> 数据类型,需要实现Serializable接口
* @author zhouhao
* @since pro 2.0
* @since 2.0
*/
public class PersistenceBuffer<T extends Serializable> implements Disposable {
public class PersistenceBuffer<T extends Serializable> implements EvictionContext, Disposable {
@SuppressWarnings("all")
private final static AtomicIntegerFieldUpdater<PersistenceBuffer> WIP =
AtomicIntegerFieldUpdater.newUpdater(PersistenceBuffer.class, "wip");
@@ -125,11 +136,16 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
//刷新缓冲区定时任务
private Disposable intervalFlush;
//独立的读写调度器
private Scheduler writer, reader;
private Throwable lastError;
private volatile Boolean disposed = false;
private boolean started = false;
private final PersistenceBufferMBeanImpl<T> monitor = new PersistenceBufferMBeanImpl<>(this);
public PersistenceBuffer(String filePath,
String fileName,
Supplier<T> newInstance,
@@ -228,6 +244,7 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
.name(fileName)
.path(path)
.option("valueType", dataType)
.option("concurrency", settings.getFileConcurrency())
.build());
this.remainder = queue.size();
//死队列,用于存放失败的数据
@@ -238,8 +255,27 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
.option("valueType", dataType)
.build());
this.deadSize = this.deadQueue.size();
this.buffer = newBuffer();
initScheduler();
registerMbean();
}
private void initScheduler() {
shutdownScheduler();
this.writer = settings.getFileConcurrency() > 1
? Schedulers.newParallel(name + "-writer", settings.getFileConcurrency())
: Schedulers.newSingle(name + "-writer");
this.reader = Schedulers.newSingle(name + "-reader");
}
private void shutdownScheduler() {
if (this.writer != null) {
this.writer.dispose();
}
if (this.reader != null) {
this.reader.dispose();
}
}
public synchronized void start() {
@@ -286,6 +322,7 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
//直接写入queue,而不是使用write,等待后续有新的数据进入再重试
if (queue.offer(buf)) {
// REMAINDER.incrementAndGet(this);
settings.getEviction().tryEviction(this);
} else {
dead(buf);
}
@@ -313,12 +350,42 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
}
return;
}
// remainder ++
monitor.in();
// REMAINDER.incrementAndGet(this);
queue.offer(data);
drain();
//尝试执行淘汰策略
settings.getEviction().tryEviction(this);
}
//异步写入数据到buffer
public Mono<Void> writeAsync(T data) {
if (isDisposed()) {
return Mono.fromRunnable(() -> write(data));
}
return Mono
.fromRunnable(() -> write(data))
.subscribeOn(writer)
.then();
}
//异步写入数据到buffer
public Mono<Void> writeAsync(Collection<T> data) {
if (isDisposed()) {
return Mono.fromRunnable(() -> data.forEach(this::write));
}
return Mono
.fromRunnable(() -> data.forEach(this::write))
.subscribeOn(writer)
.then();
}
//写入数据到buffer,此操作可能阻塞
@Deprecated
public void write(T data) {
write(new Buf<>(data, instanceBuilder));
}
@@ -328,6 +395,9 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
if (this.intervalFlush != null) {
this.intervalFlush.dispose();
}
for (FlushSubscriber subscriber : new ArrayList<>(flushing)) {
subscriber.doCancel();
}
}
@SneakyThrows
@@ -368,7 +438,9 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
deadQueue.close();
queue = null;
deadQueue = null;
shutdownScheduler();
}
unregisterMbean();
}
@Override
@@ -377,7 +449,37 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
}
public long size() {
return queue == null ? 0 : queue.size();
return queue == null || disposed ? 0 : queue.size() + buffer().size();
}
public long size(BufferType type) {
return type == BufferType.buffer ? size() : deadQueue == null || disposed ? 0 : deadQueue.size();
}
@Override
public void removeLatest(BufferType type) {
if (type == BufferType.buffer) {
if (queue.removeLast() != null) {
monitor.dropped();
}
} else {
if (deadQueue.removeLast() != null) {
// DEAD_SZIE.decrementAndGet(this);
}
}
}
@Override
public void removeOldest(BufferType type) {
if (type == BufferType.buffer) {
if (queue.removeFirst() != null) {
monitor.dropped();
}
} else {
if (deadQueue.removeFirst() != null) {
// DEAD_SZIE.decrementAndGet(this);
}
}
}
private void intervalFlush() {
@@ -446,7 +548,7 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
logger.debug("write {} data,size:{},remainder:{},requeue: {}.take up time: {} ms",
name,
buffer.size(),
queue.size(),
size(),
doRequeue,
System.currentTimeMillis() - startWith);
}
@@ -535,6 +637,7 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
}
buffer.forEach(Buf::reset);
flushing.remove(this);
monitor.out(size, System.currentTimeMillis() - startWith);
// wip--
WIP.decrementAndGet(PersistenceBuffer.this);
drain();
@@ -556,22 +659,27 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
if (!started) {
return;
}
//当前未执行完成的操作小于并行度才请求
// 当前未执行完成的操作小于并行度才请求
if (WIP.incrementAndGet(this) <= settings.getParallelism()) {
int size = settings.getBufferSize();
for (int i = 0; i < size; i++) {
if (isDisposed()) {
break;
}
Buf<T> poll = queue.poll();
if (poll != null) {
onNext(poll);
} else {
break;
}
}
// 使用boundedElastic线程执行poll,避免阻塞线程
reader
.schedule(() -> {
int size = settings.getBufferSize();
for (int i = 0; i < size && started; i++) {
Buf<T> poll = settings.getStrategy() == ConsumeStrategy.LIFO
? queue.removeLast()
: queue.poll();
if (poll != null) {
onNext(poll);
} else {
break;
}
}
WIP.decrementAndGet(this);
});
} else {
WIP.decrementAndGet(this);
}
WIP.decrementAndGet(this);
}
private void onNext(@Nonnull Buf<T> value) {
@@ -739,7 +847,7 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
if (obj.data instanceof String) {
return ((String) obj.data).length() * 2;
}
return 10_000;
return 4096;
}
@Override
@@ -802,6 +910,228 @@ public class PersistenceBuffer<T extends Serializable> implements Disposable {
}
}
private ObjectName objectName;
void registerMbean() {
try {
String safeName = name.replaceAll("[\\s\\\\/:*?\"<>|]", "_");
MBeanServer mBeanServer = ManagementFactory.getPlatformMBeanServer();
objectName = new ObjectName("org.jetlinks:type=PersistenceBuffer,name=" + safeName);
mBeanServer.registerMBean(new StandardMBean(monitor, PersistenceBufferMBean.class), objectName);
} catch (Throwable error) {
logger.warn("registerMBean {} error ", name, error);
}
}
void unregisterMbean() {
try {
if (objectName != null) {
MBeanServer mBeanServer = ManagementFactory.getPlatformMBeanServer();
mBeanServer.unregisterMBean(objectName);
}
} catch (Throwable ignore) {
}
}
@RequiredArgsConstructor
private static class PersistenceBufferMBeanImpl<T extends Serializable> implements PersistenceBufferMBean {
@SuppressWarnings("all")
private static final AtomicLongFieldUpdater<PersistenceBufferMBeanImpl>
IN = AtomicLongFieldUpdater.newUpdater(PersistenceBufferMBeanImpl.class, "in"),
OUT = AtomicLongFieldUpdater.newUpdater(PersistenceBufferMBeanImpl.class, "out"),
COST = AtomicLongFieldUpdater.newUpdater(PersistenceBufferMBeanImpl.class, "cost"),
OPT = AtomicLongFieldUpdater.newUpdater(PersistenceBufferMBeanImpl.class, "opt"),
DROPPED = AtomicLongFieldUpdater.newUpdater(PersistenceBufferMBeanImpl.class, "dropped");
private final PersistenceBuffer<T> buffer;
private volatile long
//写入数量
in,
//写出数量
out,
//写出次数
opt,
//写出总耗时
cost,
//淘汰数量
dropped;
private final long[] costDist = new long[5];
private void dropped() {
DROPPED.incrementAndGet(this);
}
private void in() {
IN.incrementAndGet(this);
}
private void out(long outSize, long cost) {
COST.addAndGet(this, cost);
OUT.addAndGet(this, outSize);
OPT.incrementAndGet(this);
if (cost < 50) {
costDist[0]++;
} else if (cost < 200) {
costDist[1]++;
} else if (cost < 1000) {
costDist[2]++;
} else if (cost < 5000) {
costDist[3]++;
} else {
costDist[4]++;
}
}
@Override
public String getMonitor() {
return String.format(
"\nqueue(in %s,out %s,dropped %s);\nconsume(opt %s,cost %s ms);\ndist[0-50ms(%s),50-200ms(%s),0.2-1s(%s),1-5s(%s),>5s(%s)]\n",
in, out, dropped, opt, cost, costDist[0], costDist[1], costDist[2], costDist[3], costDist[4]);
}
@Override
public void resetMonitor() {
IN.set(this, 0);
OUT.set(this, 0);
OPT.set(this, 0);
COST.set(this, 0);
Arrays.fill(costDist, 0);
}
@Override
public long getRemainder() {
return buffer.queue.size();
}
@Override
public long getDeadSize() {
return buffer.deadQueue.size();
}
@Override
public long getWip() {
return buffer.wip;
}
@Override
public String getLastError() {
Throwable error = buffer.lastError;
return error == null ? "nil"
: ExceptionUtils.getRootCauseMessage(error) + ":" + ExceptionUtils.getStackTrace(error);
}
@Override
public String getStoragePath() {
return buffer.settings.getFilePath();
}
@Override
public List<String> getDataBytes() {
File[] files = new File(buffer.settings.getFilePath())
.listFiles(filter -> filter.getName().startsWith(getSafeFileName(buffer.settings.getFileName())));
if (files == null) {
return Collections.emptyList();
}
return Arrays
.stream(files)
.map(file -> file.getName() + " " + FormatUtils.formatDataSize(file.length()))
.collect(Collectors.toList());
}
@Override
public void flush() {
buffer.queue.flush();
buffer.deadQueue.flush();
}
@Override
public void retryDead(int maxSize) {
//单次请求最大重试次数
maxSize = Math.min(50_0000, maxSize);
while (maxSize-- > 0) {
Buf<T> buf = buffer.deadQueue.poll();
if (buf == null) {
break;
}
buf.retry = 0;
if (!buffer.queue.offer(buf)) {
buffer.deadQueue.offer(buf);
break;
}
}
buffer.drain();
}
@Override
public String getSettings() {
return String.format("\nbufferSize: %s" +
",bufferTimeout: %s" +
",parallelism: %s" +
",maxRetryTimes: %s" +
",fileConcurrency: %s" + "\nEviction:%s ",
buffer.settings.getBufferSize(),
buffer.settings.getBufferTimeout(),
buffer.settings.getParallelism(),
buffer.settings.getMaxRetryTimes(),
buffer.settings.getFileConcurrency(),
buffer.settings.getEviction());
}
@Override
public long recovery(String fileName, boolean dead) {
return buffer.recovery(fileName, dead);
}
@Override
public List<Object> peekDead(int size) {
size = size <= 0 ? 1 : Math.min(1024, size);
List<Object> result = new ArrayList<>(size);
for (Buf<T> tBuf : buffer.deadQueue) {
if (size-- <= 0) {
break;
}
result.add(tBuf.data);
}
return result;
}
}
public interface PersistenceBufferMBean {
String getSettings();
String getMonitor();
void resetMonitor();
String getStoragePath();
long getRemainder();
long getDeadSize();
long getWip();
String getLastError();
List<String> getDataBytes();
void flush();
void retryDead(int maxSize);
long recovery(String fileName, boolean dead);
List<Object> peekDead(int size);
}
public interface FlushContext<T> {
//标记错误信息

View File

@@ -0,0 +1,37 @@
package org.jetlinks.community.buffer;
import lombok.AllArgsConstructor;
import java.util.Map;
@AllArgsConstructor
class SizeLimitEviction extends AbstractBufferEviction {
private final long bufferLimit;
private final long deadLimit;
@Override
public boolean doEviction(EvictionContext context) {
boolean anyEviction = false;
if (bufferLimit > 0 && context.size(EvictionContext.BufferType.buffer) >= bufferLimit) {
context.removeOldest(EvictionContext.BufferType.buffer);
anyEviction = true;
}
if (deadLimit > 0 && context.size(EvictionContext.BufferType.dead) >= deadLimit) {
context.removeOldest(EvictionContext.BufferType.dead);
anyEviction = true;
}
return anyEviction;
}
@Override
protected void applyEventData(Map<String, Object> data) {
data.put("bufferLimit", bufferLimit);
data.put("deadLimit", deadLimit);
}
@Override
public String toString() {
return "SizeLimit(buffer=" + bufferLimit + ", dead=" + deadLimit + ")";
}
}

View File

@@ -0,0 +1,102 @@
package org.jetlinks.community.command;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.hswebframework.web.bean.FastBeanCopier;
import org.jetlinks.core.command.CommandSupport;
import org.jetlinks.core.utils.SerializeUtils;
import org.jetlinks.community.spi.Provider;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import java.util.Map;
/**
* 命令支持提供者,用于针对多个基于命令模式的可选模块依赖时的解耦.
* <p>
*
* @author zhouhao
* @see org.jetlinks.sdk.server.SdkServices
* @see InternalSdkServices
* @since 2.1
*/
public interface CommandSupportManagerProvider {
/**
* 所有支持的提供商
*/
Provider<CommandSupportManagerProvider> supports = Provider.create(CommandSupportManagerProvider.class);
/**
* 命令服务提供商标识
*
* @return 唯一标识
*/
String getProvider();
/**
* 获取命令支持,不同的命令管理支持多种命令支持,可能通过id进行区分,具体规则由对应服务实 现
*
* @param id 命令ID标识
* @param options 拓展配置
* @return CommandSupport
* @see CommandSupportManagerProviders#getCommandSupport(String, Map)
*/
Mono<? extends CommandSupport> getCommandSupport(String id, Map<String, Object> options);
/**
* 获取所有支持的信息
*
* @return id
*/
default Flux<CommandSupportInfo> getSupportInfo() {
return Flux.empty();
}
@Getter
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
@Setter
class CommandSupportInfo implements Externalizable {
private String id;
private String name;
private String description;
public CommandSupportInfo copy() {
return FastBeanCopier.copy(this, new CommandSupportInfo());
}
/**
* @param serviceId serviceId
* @return this
* @see CommandSupportManagerProviders#getCommandSupport(String)
*/
public CommandSupportInfo appendService(String serviceId) {
if (this.id == null) {
this.id = serviceId;
} else {
this.id = serviceId + ":" + id;
}
return this;
}
@Override
public void writeExternal(ObjectOutput out) throws IOException {
SerializeUtils.writeNullableUTF(id, out);
SerializeUtils.writeNullableUTF(name, out);
SerializeUtils.writeNullableUTF(description, out);
}
@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
id = SerializeUtils.readNullableUTF(in);
name = SerializeUtils.readNullableUTF(in);
description = SerializeUtils.readNullableUTF(in);
}
}
}

View File

@@ -0,0 +1,121 @@
package org.jetlinks.community.command;
import org.jetlinks.core.command.CommandSupport;
import reactor.core.publisher.Mono;
import java.util.Collections;
import java.util.Map;
import java.util.Optional;
/**
* 命令支持管理提供商工具类,用于提供对{@link CommandSupportManagerProvider}相关通用操作.
*
* @author zhouhao
* @see CommandSupportManagerProvider
* @see CommandSupportManagerProviders#getCommandSupport(String, Map)
* @since 2.2
*/
public class CommandSupportManagerProviders {
/**
* 根据服务ID获取CommandSupport.
* <pre>{@code
*
* CommandSupportManagerProviders
* .getCommandSupport("deviceService:device",Collections.emptyMap())
*
* }</pre>
*
* @param serviceId serviceId 服务名
* @return CommandSupport
* @see InternalSdkServices
* @see org.jetlinks.sdk.server.SdkServices
*/
public static Mono<CommandSupport> getCommandSupport(String serviceId) {
return getCommandSupport(serviceId, Collections.emptyMap());
}
/**
* 根据服务ID和支持ID获取CommandSupport.
*
* @param serviceId 服务ID
* @param supportId 支持ID
* @return CommandSupport
*/
public static Mono<CommandSupport> getCommandSupport(String serviceId, String supportId) {
return getProviderNow(serviceId)
.getCommandSupport(supportId, Collections.emptyMap())
.cast(CommandSupport.class);
}
/**
* 根据服务ID获取CommandSupport.
* <pre>{@code
*
* CommandSupportManagerProviders
* .getCommandSupport("deviceService:device",Collections.emptyMap())
*
* }</pre>
*
* @param serviceId serviceId 服务名
* @param options options
* @return CommandSupport
* @see InternalSdkServices
* @see org.jetlinks.sdk.server.SdkServices
*/
public static Mono<CommandSupport> getCommandSupport(String serviceId,
Map<String, Object> options) {
//fast path
CommandSupportManagerProvider provider = CommandSupportManagerProvider
.supports
.get(serviceId)
.orElse(null);
if (provider != null) {
return provider
.getCommandSupport(serviceId, options)
.cast(CommandSupport.class);
}
String supportId = serviceId;
// deviceService:product
if (serviceId.contains(":")) {
String[] arr = serviceId.split(":", 2);
serviceId = arr[0];
supportId = arr[1];
}
String finalServiceId = serviceId;
String finalSupportId = supportId;
return Mono.defer(() -> getProviderNow(finalServiceId).getCommandSupport(finalSupportId, options));
}
/**
* 注册命令支持
*
* @param provider {@link CommandSupportManagerProvider#getProvider()}
*/
public static void register(CommandSupportManagerProvider provider) {
CommandSupportManagerProvider.supports.register(provider.getProvider(), provider);
}
/**
* 获取命令支持
*
* @param provider {@link CommandSupportManagerProvider#getProvider()}
* @return Optional
*/
public static Optional<CommandSupportManagerProvider> getProvider(String provider) {
return CommandSupportManagerProvider.supports.get(provider);
}
/**
* 获取命令支持,如果不存在则抛出异常{@link UnsupportedOperationException}
*
* @param provider provider {@link CommandSupportManagerProvider#getProvider()}
* @return CommandSupportManagerProvider
*/
public static CommandSupportManagerProvider getProviderNow(String provider) {
return CommandSupportManagerProvider.supports.getNow(provider);
}
}

View File

@@ -0,0 +1,37 @@
package org.jetlinks.community.command;
import lombok.AllArgsConstructor;
import org.jetlinks.core.command.CommandSupport;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.util.List;
import java.util.Map;
@AllArgsConstructor
public class CompositeCommandSupportManagerProvider implements CommandSupportManagerProvider {
private final List<CommandSupportManagerProvider> providers;
@Override
public String getProvider() {
return providers.get(0).getProvider();
}
@Override
public Mono<? extends CommandSupport> getCommandSupport(String id, Map<String, Object> options) {
return Flux
.fromIterable(providers)
.flatMap(provider -> provider.getCommandSupport(id, options))
.take(1)
.singleOrEmpty();
}
@Override
public Flux<CommandSupportInfo> getSupportInfo() {
return Flux
.fromIterable(providers)
.flatMap(CommandSupportManagerProvider::getSupportInfo)
.distinct(CommandSupportInfo::getId);
}
}

View File

@@ -0,0 +1,307 @@
package org.jetlinks.community.command;
import lombok.SneakyThrows;
import org.hswebframework.web.api.crud.entity.EntityFactoryHolder;
import org.hswebframework.web.authorization.Authentication;
import org.hswebframework.web.authorization.Permission;
import org.hswebframework.web.authorization.exception.AccessDenyException;
import org.hswebframework.web.bean.FastBeanCopier;
import org.hswebframework.web.crud.service.ReactiveCrudService;
import org.jetlinks.core.command.AbstractCommandSupport;
import org.jetlinks.core.command.Command;
import org.jetlinks.core.metadata.FunctionMetadata;
import org.jetlinks.core.metadata.SimplePropertyMetadata;
import org.jetlinks.core.metadata.types.ArrayType;
import org.jetlinks.core.metadata.types.IntType;
import org.jetlinks.core.metadata.types.ObjectType;
import org.jetlinks.core.metadata.types.StringType;
import org.jetlinks.core.utils.Reactors;
import org.jetlinks.sdk.server.commons.cmd.*;
import org.jetlinks.supports.official.DeviceMetadataParser;
import org.springframework.core.ResolvableType;
import reactor.bool.BooleanUtils;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import javax.annotation.Nonnull;
import javax.annotation.Nullable;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
/**
* 通用增删改查命令支持,基于{@link ReactiveCrudService}来实现增删改查相关命令
*
* @param <T> 实体类型
* @author zhouhao
* @see QueryByIdCommand
* @see QueryPagerCommand
* @see QueryListCommand
* @see CountCommand
* @see SaveCommand
* @see AddCommand
* @see UpdateCommand
* @see DeleteCommand
* @see DeleteByIdCommand
* @since 2.2
*/
public class CrudCommandSupport<T> extends AbstractCommandSupport {
final ReactiveCrudService<T, String> service;
final ResolvableType _entityType;
public CrudCommandSupport(ReactiveCrudService<T, String> service) {
this(service, ResolvableType
.forClass(ReactiveCrudService.class, service.getClass())
.getGeneric(0));
}
public CrudCommandSupport(ReactiveCrudService<T, String> service, ResolvableType _entityType) {
this.service = service;
this._entityType = _entityType;
registerQueries();
registerSaves();
registerDelete();
}
@Override
public Flux<FunctionMetadata> getCommandMetadata() {
return super
.getCommandMetadata()
.filterWhen(func -> commandIsSupported(func.getId()));
}
@Override
public Mono<Boolean> commandIsSupported(String commandId) {
return BooleanUtils.and(
super.commandIsSupported(commandId),
hasPermission(getPermissionId(), getAction(commandId))
);
}
@Override
public Mono<FunctionMetadata> getCommandMetadata(String commandId) {
return super
.getCommandMetadata(commandId)
.filterWhen(func -> commandIsSupported(func.getId()));
}
@Override
public Mono<FunctionMetadata> getCommandMetadata(Command<?> command) {
return super
.getCommandMetadata(command)
.filterWhen(func -> commandIsSupported(func.getId()));
}
@Override
public Mono<FunctionMetadata> getCommandMetadata(@Nonnull String commandId,
@Nullable Map<String, Object> parameters) {
return super
.getCommandMetadata(commandId, parameters)
.filterWhen(func -> commandIsSupported(func.getId()));
}
@SneakyThrows
@SuppressWarnings("all")
private T newInstance0() {
return (T) _entityType.toClass().getConstructor().newInstance();
}
protected T newInstance() {
@SuppressWarnings("all")
Class<T> clazz = (Class<T>) _entityType.toClass();
return EntityFactoryHolder
.newInstance(clazz,
this::newInstance0);
}
protected ResolvableType getResolvableType() {
return _entityType;
}
protected ObjectType createEntityType() {
return (ObjectType) DeviceMetadataParser.withType(_entityType);
}
protected String getPermissionId() {
return null;
}
protected Mono<Void> assetPermission(String action) {
return assetPermission(getPermissionId(), action);
}
protected Mono<Boolean> hasPermission(String permissionId, String action) {
if (permissionId == null) {
return Reactors.ALWAYS_TRUE;
}
return Authentication
.currentReactive()
.map(auth -> auth.hasPermission(permissionId, action))
.defaultIfEmpty(true);
}
protected Mono<Void> assetPermission(String permissionId, String action) {
if (permissionId == null) {
return Mono.empty();
}
return Authentication
.currentReactive()
.flatMap(
auth -> auth.hasPermission(permissionId, action)
? Mono.empty()
: Mono.error(new AccessDenyException.NoStackTrace(permissionId, Collections.singleton(action))));
}
protected String getAction(String commandId) {
if (commandId.startsWith("Delete")) {
return Permission.ACTION_DELETE;
}
if (commandId.startsWith("Update") ||
commandId.startsWith("Save") ||
commandId.startsWith("Add") ||
commandId.startsWith("Disable") ||
commandId.startsWith("Enable")) {
return Permission.ACTION_SAVE;
}
return Permission.ACTION_QUERY;
}
protected void registerQueries() {
//根据id查询
registerHandler(
QueryByIdCommand
.<T>createHandler(
metadata -> {
metadata
.setInputs(Collections.singletonList(
SimplePropertyMetadata
.of("id", "id", new ArrayType()
.elementType(StringType.GLOBAL))
));
metadata.setOutput(createEntityType());
},
cmd -> assetPermission(Permission.ACTION_QUERY)
.then(service.findById(cmd.getId())),
_entityType)
);
//分页查询
registerHandler(
QueryPagerCommand
.<T>createHandler(
metadata -> metadata.setOutput(
QueryPagerCommand
.createOutputType(createEntityType().getProperties())),
cmd -> assetPermission(Permission.ACTION_QUERY)
.then(service.queryPager(cmd.asQueryParam())),
_entityType)
);
//查询列表
registerHandler(
QueryListCommand
.<T>createHandler(
metadata -> metadata.setOutput(createEntityType()),
cmd -> assetPermission(Permission.ACTION_QUERY)
.thenMany(service.query(cmd.asQueryParam())),
_entityType)
);
//查询数量
registerHandler(
CountCommand
.createHandler(
metadata -> metadata.setOutput(new ObjectType()
.addProperty("total", "总数", IntType.GLOBAL)),
cmd -> assetPermission(Permission.ACTION_QUERY)
.then(service.count(cmd.asQueryParam())))
);
//todo 聚合查询?
}
protected void registerSaves() {
//批量保存
registerHandler(
SaveCommand.createHandler(
metadata -> {
metadata
.setInputs(Collections.singletonList(
SimplePropertyMetadata.of("data", "数据列表", new ArrayType().elementType(createEntityType()))
));
metadata.setOutput(createEntityType());
},
cmd -> {
List<T> list = cmd.dataList((data) -> FastBeanCopier.copy(data, newInstance()));
return assetPermission(Permission.ACTION_SAVE)
.then(service.save(list))
.thenMany(Flux.fromIterable(list));
},
_entityType)
);
//新增
registerHandler(
AddCommand
.<T>createHandler(
metadata -> {
metadata
.setInputs(Collections.singletonList(
SimplePropertyMetadata.of("data", "数据列表", new ArrayType().elementType(createEntityType()))
));
metadata.setOutput(createEntityType());
},
cmd -> Flux
.fromIterable(cmd.dataList((data) -> FastBeanCopier.copy(data, newInstance())))
.as(flux -> assetPermission(Permission.ACTION_SAVE)
.then(service.insert(flux))
.thenMany(flux)))
);
//修改
registerHandler(
UpdateCommand
.<T>createHandler(
metadata -> {
metadata.setInputs(
Arrays.asList(
SimplePropertyMetadata.of("data", "数据", createEntityType()),
QueryCommand.getTermsMetadata()
));
metadata.setOutput(IntType.GLOBAL);
},
cmd -> this
.assetPermission(Permission.ACTION_SAVE)
.then(cmd
.applyUpdate(service.createUpdate(), map -> FastBeanCopier.copy(map, newInstance()))
.execute()))
);
}
protected void registerDelete() {
//删除
registerHandler(
DeleteCommand.createHandler(
metadata -> {
metadata.setInputs(Collections.singletonList(
SimplePropertyMetadata.of("terms", "删除条件", QueryCommand.getTermsDataType())));
metadata.setOutput(IntType.GLOBAL);
},
cmd -> this
.assetPermission(Permission.ACTION_DELETE)
.then(cmd.applyDelete(service.createDelete()).execute()))
);
//根据id移除
registerHandler(
DeleteByIdCommand
.<Mono<Void>>createHandler(
metadata -> {
},
cmd -> this
.assetPermission(Permission.ACTION_DELETE)
.then(service.deleteById(cmd.getId()).then()))
);
}
}

View File

@@ -0,0 +1,43 @@
package org.jetlinks.community.command;
/**
* 平台内部的一些服务定义
*
* @author zhouhao
* @see org.jetlinks.sdk.server.SdkServices
* @since 2.2
*/
public interface InternalSdkServices {
/**
* 网络组件服务
*/
String networkService = "networkService";
/**
* 设备接入网关服务
*/
String deviceGatewayService = "deviceGatewayService";
/**
* 采集器服务
*/
String collectorService = "collectorService";
/**
* 规则服务
*/
String ruleService = "ruleService";
/**
* 插件服务
*/
String pluginService = "pluginService";
/**
* 基础服务
*/
String commonService = "commonService";
}

View File

@@ -0,0 +1,75 @@
package org.jetlinks.community.command;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.hswebframework.web.i18n.LocaleUtils;
import org.jetlinks.core.command.AbstractCommandSupport;
import org.jetlinks.core.command.CommandSupport;
import org.jetlinks.community.annotation.command.CommandService;
import org.springframework.core.annotation.AnnotationUtils;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.util.HashMap;
import java.util.Map;
/**
* 通用静态命令管理.
*
* @author zhangji 2024/2/2
* @since 2.2.0
*/
@AllArgsConstructor
public class StaticCommandSupportManagerProvider extends AbstractCommandSupport implements CommandSupportManagerProvider {
@Getter
public String provider;
private final Map<String, CommandSupport> commandSupports = new HashMap<>();
public void register(String id, CommandSupport commandSupport) {
commandSupports.put(id, commandSupport);
}
@Override
public final Mono<? extends CommandSupport> getCommandSupport(String id, Map<String, Object> options) {
CommandSupport cmd = commandSupports.get(id);
if (cmd == null) {
return getUndefined(id, options);
}
return Mono.just(cmd);
}
protected Mono<? extends CommandSupport> getUndefined(String id, Map<String, Object> options) {
return Mono.just(this);
}
@Override
public Flux<CommandSupportInfo> getSupportInfo() {
Flux<CommandSupportInfo> another = Flux
.fromIterable(commandSupports.entrySet())
.map(entry -> createCommandSupport(entry.getKey(), entry.getValue().getClass()));
if (!this.handlers.isEmpty()) {
return Flux.concat(another, Flux.just(createCommandSupport(null, this.getClass())));
}
return another;
}
protected final CommandSupportInfo createCommandSupport(String id, Class<?> clazz) {
String name = id;
String description = null;
Schema schema = AnnotationUtils.findAnnotation(clazz, Schema.class);
if (null != schema) {
name = schema.title();
description = schema.description();
}
CommandService service = AnnotationUtils.findAnnotation(clazz, CommandService.class);
if (null != service) {
name = LocaleUtils.resolveMessage(service.name(), service.name());
description = String.join("", service.description());
}
return CommandSupportInfo.of(id, name, description);
}
}

View File

@@ -0,0 +1,83 @@
package org.jetlinks.community.command.register;
import lombok.extern.slf4j.Slf4j;
import org.jetlinks.community.command.CommandSupportManagerProvider;
import org.jetlinks.community.command.CommandSupportManagerProviders;
import org.jetlinks.community.command.CompositeCommandSupportManagerProvider;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.SmartInitializingSingleton;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.core.annotation.AnnotatedElementUtils;
import org.springframework.util.ClassUtils;
import org.jetlinks.community.annotation.command.CommandService;
import javax.annotation.Nonnull;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
@Slf4j
public class CommandServiceEndpointRegister implements ApplicationContextAware, SmartInitializingSingleton {
private ApplicationContext context;
@Override
public void setApplicationContext(@Nonnull ApplicationContext applicationContext) throws BeansException {
this.context = applicationContext;
}
@Override
public void afterSingletonsInstantiated() {
Map<String, Object> beans = context.getBeansWithAnnotation(CommandService.class);
//静态Provider
Map<String, List<CommandSupportManagerProvider>> statics = context
.getBeanProvider(CommandSupportManagerProvider.class)
.stream()
.collect(Collectors.groupingBy(CommandSupportManagerProvider::getProvider));
Map<String, SpringBeanCommandSupportProvider> providers = new HashMap<>();
for (Object value : beans.values()) {
CommandService endpoint =
AnnotatedElementUtils.findMergedAnnotation(ClassUtils.getUserClass(value), CommandService.class);
if (endpoint == null || !endpoint.autoRegistered()) {
continue;
}
String id = endpoint.id();
String support = id;
if (id.contains(":")) {
support = id.substring(id.indexOf(":") + 1);
id = id.substring(0, id.indexOf(":"));
}
SpringBeanCommandSupportProvider provider = providers
.computeIfAbsent(id, SpringBeanCommandSupportProvider::new);
log.debug("register command support:{} -> {}", endpoint.id(), value);
provider.register(support, endpoint, value);
}
for (SpringBeanCommandSupportProvider value : providers.values()) {
if (value.isEmpty()) {
continue;
}
//合并静态Provider
List<CommandSupportManagerProvider> provider = statics.remove(value.getProvider());
if (provider != null) {
provider.forEach(value::register);
}
CommandSupportManagerProviders.register(value);
}
for (List<CommandSupportManagerProvider> value : statics.values()) {
if (value.size() == 1) {
CommandSupportManagerProviders.register(value.get(0));
} else {
CommandSupportManagerProviders.register(new CompositeCommandSupportManagerProvider(value));
}
}
}
}

View File

@@ -0,0 +1,138 @@
package org.jetlinks.community.command.register;
import com.google.common.collect.Lists;
import lombok.Getter;
import org.hswebframework.web.i18n.LocaleUtils;
import org.jetlinks.core.command.CommandSupport;
import org.jetlinks.core.command.CompositeCommandSupport;
import org.jetlinks.community.annotation.command.CommandService;
import org.jetlinks.community.command.CommandSupportManagerProvider;
import org.jetlinks.supports.command.JavaBeanCommandSupport;
import org.springframework.util.StringUtils;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.util.*;
class SpringBeanCommandSupportProvider implements CommandSupportManagerProvider {
private final String provider;
private final Map<String, CompositeSpringBeanCommandSupport> commandSupports = new HashMap<>();
private final List<CommandSupportManagerProvider> statics = new ArrayList<>();
public SpringBeanCommandSupportProvider(String provider) {
this.provider = provider;
}
void register(CommandSupportManagerProvider provider) {
statics.add(provider);
}
void register(String support, CommandService annotation, Object bean) {
Objects.requireNonNull(annotation, "endpoint");
Objects.requireNonNull(bean, "bean");
SpringBeanCommandSupport commandSupport = new SpringBeanCommandSupport(annotation, bean);
if (commandSupport.isEmpty()) {
return;
}
//相同support合并成一个
commandSupports
.computeIfAbsent(support, id -> new CompositeSpringBeanCommandSupport(id,provider))
.register(commandSupport);
}
boolean isEmpty() {
return commandSupports.isEmpty();
}
@Override
public String getProvider() {
return provider;
}
@Override
public Mono<? extends CommandSupport> getCommandSupport(String id, Map<String, Object> options) {
CommandSupport support = commandSupports.get(StringUtils.hasText(id) ? id : provider);
if (support != null) {
return Mono.just(support);
}
if (statics.isEmpty()) {
return Mono.empty();
}
return Flux
.fromIterable(statics)
.flatMap(provider -> provider.getCommandSupport(id, options))
.take(1)
.singleOrEmpty();
}
@Override
public Flux<CommandSupportInfo> getSupportInfo() {
if (statics.isEmpty()) {
return Flux
.fromIterable(commandSupports.values())
.flatMapIterable(CompositeSpringBeanCommandSupport::getInfo);
}
return Flux
.concat(
Flux
.fromIterable(commandSupports.values())
.flatMapIterable(CompositeSpringBeanCommandSupport::getInfo),
Flux.fromIterable(statics)
.flatMap(CommandSupportManagerProvider::getSupportInfo)
)
.distinct(info -> {
String id = info.getId();
return String.valueOf(id);
});
}
static class CompositeSpringBeanCommandSupport extends CompositeCommandSupport {
private final String id;
private final String provider;
public CompositeSpringBeanCommandSupport(String id,String provider) {
super();
this.id = id;
this.provider = provider;
}
public List<CommandSupportInfo> getInfo() {
return Lists
.transform(
getSupports(),
support -> {
SpringBeanCommandSupport commandSupport = support.unwrap(SpringBeanCommandSupport.class);
//兼容为null
String _id = id.equals(provider) ? null : id;
return CommandSupportInfo.of(
_id,
LocaleUtils.resolveMessage(commandSupport.annotation.name(), commandSupport.annotation.name()),
String.join("", commandSupport.annotation.description())
);
});
}
@Override
public String toString() {
return getSupports().toString();
}
}
@Getter
static class SpringBeanCommandSupport extends JavaBeanCommandSupport {
private final CommandService annotation;
boolean isEmpty() {
return handlers.isEmpty();
}
public SpringBeanCommandSupport(CommandService annotation, Object target) {
super(target);
this.annotation = annotation;
}
}
}

View File

@@ -0,0 +1,35 @@
package org.jetlinks.community.command.rule;
import io.swagger.v3.oas.annotations.media.Schema;
import org.hswebframework.web.bean.FastBeanCopier;
import org.jetlinks.core.command.AbstractCommand;
import org.jetlinks.core.command.CommandMetadataResolver;
import org.jetlinks.core.command.CommandUtils;
import org.jetlinks.core.metadata.FunctionMetadata;
import org.jetlinks.core.metadata.SimpleFunctionMetadata;
import org.jetlinks.community.command.rule.data.RelieveInfo;
import org.jetlinks.community.command.rule.data.RelieveResult;
import org.springframework.core.ResolvableType;
import reactor.core.publisher.Mono;
@Schema(title = "解除告警命令")
public class RelievedAlarmCommand extends AbstractCommand<Mono<RelieveResult>,RelievedAlarmCommand> {
@Schema(description = "解除告警传参信息")
public RelieveInfo getRelieveInfo() {
return FastBeanCopier.copy(readable(), new RelieveInfo());
}
public RelievedAlarmCommand setRelieveInfo(RelieveInfo relieveInfo) {
return with(FastBeanCopier.copy(relieveInfo, writable()));
}
public static FunctionMetadata metadata() {
SimpleFunctionMetadata metadata = new SimpleFunctionMetadata();
metadata.setId(CommandUtils.getCommandIdByType(RelievedAlarmCommand.class));
metadata.setName("解除告警命令");
metadata.setInputs(CommandMetadataResolver.resolveInputs(ResolvableType.forClass(RelieveInfo.class)));
metadata.setOutput(CommandMetadataResolver.resolveOutput(ResolvableType.forClass(RelievedAlarmCommand.class)));
return metadata;
}
}

View File

@@ -0,0 +1,34 @@
package org.jetlinks.community.command.rule;
public interface RuleCommandServices {
/**
* 场景
*/
String sceneService = "sceneService";
/**
* 告警配置
*/
String alarmConfigService = "alarmConfigService";
/**
* 告警记录
*/
String alarmRecordService = "alarmRecordService";
/**
* 告警历史
*/
String alarmHistoryService = "alarmHistoryService";
/**
* 告警规则绑定
*/
String alarmRuleBindService = "alarmRuleBindService";
/**
* 告警相关
*/
String alarm = "alarm";
}

View File

@@ -0,0 +1,38 @@
package org.jetlinks.community.command.rule;
import io.swagger.v3.oas.annotations.media.Schema;
import org.hswebframework.web.bean.FastBeanCopier;
import org.jetlinks.core.command.AbstractCommand;
import org.jetlinks.core.command.CommandMetadataResolver;
import org.jetlinks.core.command.CommandUtils;
import org.jetlinks.core.metadata.FunctionMetadata;
import org.jetlinks.core.metadata.SimpleFunctionMetadata;
import org.jetlinks.community.command.rule.data.AlarmInfo;
import org.jetlinks.community.command.rule.data.AlarmResult;
import org.springframework.core.ResolvableType;
import reactor.core.publisher.Mono;
@Schema(title = "触发告警命令")
public class TriggerAlarmCommand extends AbstractCommand<Mono<AlarmResult>,TriggerAlarmCommand> {
private static final long serialVersionUID = 7056867872399432831L;
@Schema(description = "告警传参信息")
public AlarmInfo getAlarmInfo() {
return FastBeanCopier.copy(readable(), new AlarmInfo());
}
public TriggerAlarmCommand setAlarmInfo(AlarmInfo alarmInfo) {
return with(FastBeanCopier.copy(alarmInfo, writable()));
}
public static FunctionMetadata metadata() {
SimpleFunctionMetadata metadata = new SimpleFunctionMetadata();
metadata.setId(CommandUtils.getCommandIdByType(TriggerAlarmCommand.class));
metadata.setName("触发告警命令");
metadata.setInputs(CommandMetadataResolver.resolveInputs(ResolvableType.forClass(AlarmInfo.class)));
metadata.setOutput(CommandMetadataResolver.resolveOutput(ResolvableType.forClass(TriggerAlarmCommand.class)));
return metadata;
}
}

View File

@@ -9,7 +9,6 @@ import lombok.Setter;
import org.jetlinks.community.terms.TermSpec;
import java.io.Serializable;
import java.util.List;
import java.util.Map;
/**
@@ -49,6 +48,9 @@ public class AlarmInfo implements Serializable {
@Schema(description = "告警来源ID")
private String sourceId;
@Schema(description = "告警来源的创建人ID")
private String sourceCreatorId;
@Schema(description = "告警来源名称")
private String sourceName;
@@ -65,4 +67,8 @@ public class AlarmInfo implements Serializable {
* 告警触发条件
*/
private TermSpec termSpec;
@Schema(description = "告警时间")
private Long alarmTime;
}

View File

@@ -18,6 +18,9 @@ public class RelieveInfo extends AlarmInfo{
@Schema(description = "解除原因")
private String relieveReason;
@Schema(description = "解除时间")
private Long relieveTime;
@Schema(description = "解除说明")
private String describe;

View File

@@ -7,19 +7,21 @@ import lombok.Generated;
import org.apache.commons.beanutils.BeanUtilsBean;
import org.apache.commons.beanutils.Converter;
import org.hswebframework.ezorm.rdb.mapping.ReactiveRepository;
import org.hswebframework.web.api.crud.entity.EntityFactory;
import org.hswebframework.web.bean.FastBeanCopier;
import org.hswebframework.web.cache.ReactiveCacheManager;
import org.hswebframework.web.dict.EnumDict;
import org.hswebframework.web.dict.defaults.DefaultItemDefine;
import org.jetlinks.community.Interval;
import org.jetlinks.community.JvmErrorException;
import org.jetlinks.community.command.register.CommandServiceEndpointRegister;
import org.jetlinks.community.config.ConfigManager;
import org.jetlinks.community.config.ConfigScopeCustomizer;
import org.jetlinks.community.config.ConfigScopeProperties;
import org.jetlinks.community.config.SimpleConfigManager;
import org.jetlinks.community.config.entity.ConfigEntity;
import org.jetlinks.community.dictionary.DictionaryJsonDeserializer;
import org.jetlinks.community.reactorql.aggregation.InternalAggregationSupports;
import org.jetlinks.community.reactorql.function.InternalFunctionSupport;
import org.jetlinks.community.reference.DataReferenceManager;
import org.jetlinks.community.reference.DataReferenceProvider;
import org.jetlinks.community.reference.DefaultDataReferenceManager;
@@ -30,12 +32,12 @@ import org.jetlinks.community.resource.TypeScriptDeclareResourceProvider;
import org.jetlinks.community.resource.initialize.PermissionResourceProvider;
import org.jetlinks.community.service.DefaultUserBindService;
import org.jetlinks.community.utils.TimeUtils;
import org.jetlinks.core.event.EventBus;
import org.jetlinks.core.metadata.DataType;
import org.jetlinks.core.rpc.RpcManager;
import org.jetlinks.core.metadata.types.DataTypes;
import org.jetlinks.reactor.ql.feature.Feature;
import org.jetlinks.reactor.ql.supports.DefaultReactorQLMetadata;
import org.jetlinks.reactor.ql.utils.CastUtils;
import org.jetlinks.supports.official.JetLinksDataTypeCodecs;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.beans.factory.config.BeanPostProcessor;
@@ -47,25 +49,26 @@ import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.core.ReactiveRedisOperations;
import org.springframework.http.MediaType;
import org.springframework.util.StringUtils;
import org.springframework.util.unit.DataSize;
import reactor.core.Exceptions;
import reactor.core.publisher.Hooks;
import javax.annotation.Nonnull;
import java.time.Duration;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.LocalTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.Date;
import java.util.Map;
@Configuration
@AutoConfiguration
@SuppressWarnings("all")
@EnableConfigurationProperties({ConfigScopeProperties.class})
public class CommonConfiguration {
static {
InternalAggregationSupports.register();
InternalFunctionSupport.register();
BeanUtilsBean.getInstance().getConvertUtils().register(new Converter() {
@Override
public <T> T convert(Class<T> aClass, Object o) {
@@ -149,6 +152,23 @@ public class CommonConfiguration {
}
}, EnumDict.class);
BeanUtilsBean.getInstance().getConvertUtils().register(new Converter() {
@Override
@Generated
public <T> T convert(Class<T> type, Object value) {
if (value instanceof Map) {
Map<String, Object> map = ((Map) value);
String typeId = (String) map.get("type");
if (StringUtils.isEmpty(typeId)) {
return null;
}
return (T) JetLinksDataTypeCodecs.decode(DataTypes.lookup(typeId).get(), map);
}
return null;
}
}, DataType.class);
//捕获jvm错误,防止Flux被挂起
Hooks.onOperatorError((err, val) -> {
if (Exceptions.isJvmFatal(err)) {
@@ -180,6 +200,7 @@ public class CommonConfiguration {
@Bean
public Jackson2ObjectMapperBuilderCustomizer jackson2ObjectMapperBuilderCustomizer(){
return builder->{
builder.deserializerByType(DataType.class, new DataTypeJSONDeserializer());
builder.deserializerByType(Date.class,new SmartDateDeserializer());
builder.deserializerByType(EnumDict.class, new DictionaryJsonDeserializer());
};
@@ -223,6 +244,12 @@ public class CommonConfiguration {
return referenceManager;
}
@Bean
public CommandServiceEndpointRegister commandServiceEndpointRegister() {
return new CommandServiceEndpointRegister();
}
@Configuration
@ConditionalOnClass(ReactiveRedisOperations.class)
static class DefaultUserBindServiceConfiguration {

View File

@@ -0,0 +1,26 @@
package org.jetlinks.community.configuration;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import org.apache.commons.beanutils.BeanUtilsBean;
import org.jetlinks.core.metadata.DataType;
import java.io.IOException;
import java.util.Map;
/**
*
* @author zhangji 2025/1/23
* @since 2.3
*/
public class DataTypeJSONDeserializer extends JsonDeserializer<DataType> {
@Override
public DataType deserialize(JsonParser parser, DeserializationContext ctxt) throws IOException, JsonProcessingException {
Map<String,Object> map= ctxt.readValue(parser, Map.class);
return (DataType) BeanUtilsBean.getInstance().getConvertUtils().convert(map, DataType.class);
}
}

View File

@@ -0,0 +1,24 @@
package org.jetlinks.community.configuration;
import org.jetlinks.community.resource.ui.UiMenuResourceProvider;
import org.jetlinks.community.resource.ui.UiResourceProvider;
import org.springframework.boot.autoconfigure.AutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
@AutoConfiguration
@ConditionalOnProperty(prefix = "jetlinks.ui", name = "enabled", havingValue = "true", matchIfMissing = true)
public class UiResourceConfiguration {
@Bean
public UiResourceProvider uiResourceProvider() {
return new UiResourceProvider();
}
@Bean
public UiMenuResourceProvider uiMenuResourceProvider() {
return new UiMenuResourceProvider();
}
}

View File

@@ -0,0 +1,15 @@
package org.jetlinks.community.event;
import org.jetlinks.community.Operation;
import org.jetlinks.community.OperationType;
import reactor.core.publisher.Flux;
public interface OperationAssetProvider {
OperationType[] getSupportTypes();
Flux<String> createTopics(Operation operation, String original);
Flux<String> getAssetTypes(String operationType);
}

View File

@@ -0,0 +1,35 @@
package org.jetlinks.community.event;
import lombok.extern.slf4j.Slf4j;
import org.jetlinks.community.Operation;
import org.jetlinks.community.OperationType;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
@Slf4j
public class OperationAssetProviders {
private static final Map<String, OperationAssetProvider> providers = new ConcurrentHashMap<>();
public static void register(OperationAssetProvider provider) {
for (OperationType supportType : provider.getSupportTypes()) {
OperationAssetProvider old = providers.put(supportType.getId(), provider);
if (old != null && old != provider) {
log.warn("operation asset provider [{}] already exists,will be replaced by [{}]", old, provider);
}
}
}
public static Optional<OperationAssetProvider> lookup(Operation operation) {
return lookup(operation.getType().getId());
}
public static Optional<OperationAssetProvider> lookup(String operationType) {
return Optional.ofNullable(providers.get(operationType));
}
}

View File

@@ -0,0 +1,77 @@
package org.jetlinks.community.event;
import lombok.Getter;
import lombok.Setter;
import org.jetlinks.core.utils.SerializeUtils;
import org.jetlinks.community.Operation;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
@Getter
@Setter
public class SystemEvent implements Externalizable {
private static final long serialVersionUID = 1L;
private Level level;
private String code;
private Operation operation;
/**
* 描述详情,不同的类型详情内容不同
*
* @see org.jetlinks.community.monitor.ExecutionMonitorInfo
*/
private Object detail;
private long timestamp;
public SystemEvent(Level level, String code, Operation operation, Object detail) {
this.level = level;
this.code = code;
this.operation = operation;
this.detail = detail;
this.timestamp = System.currentTimeMillis();
}
public SystemEvent() {
}
@Override
public void writeExternal(ObjectOutput out) throws IOException {
out.writeByte(level.ordinal());
out.writeUTF(code);
operation.writeExternal(out);
SerializeUtils.writeObject(detail, out);
out.writeLong(timestamp);
}
@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
level = Level.values()[in.readByte()];
code = in.readUTF();
operation = new Operation();
operation.readExternal(in);
detail = SerializeUtils.readObject(in);
timestamp = in.readLong();
}
public enum Level {
info,
warn,
error
}
}

View File

@@ -0,0 +1,48 @@
package org.jetlinks.community.event;
import lombok.AllArgsConstructor;
import org.jetlinks.core.event.EventBus;
import org.springframework.context.ApplicationEventPublisher;
/**
* 推送系统事件到事件总线topic: /sys-event/{operationType}/{operationId}/{level}
*
* @author zhouhao
* @since 2.0
*/
@AllArgsConstructor
public class SystemEventDispatcher implements SystemEventHandler {
private final EventBus eventBus;
private final ApplicationEventPublisher eventPublisher;
@Override
public final void handle(SystemEvent event) {
String topic = SystemEventHandler
.topic(event.getOperation().getType().getId(),
event.getOperation().getSource().getId(),
event.getLevel().name());
eventPublisher.publishEvent(event);
OperationAssetProvider provider = OperationAssetProviders
.lookup(event.getOperation())
.orElse(null);
//对数据权限控制的支持
if (provider != null) {
provider
.createTopics(event.getOperation(), topic)
.flatMap(_topic -> eventBus.publish(_topic, event))
.subscribe();
} else {
eventBus.publish(topic, event)
.subscribe();
}
}
}

View File

@@ -0,0 +1,22 @@
package org.jetlinks.community.event;
import org.jetlinks.core.utils.StringBuilderUtils;
public interface SystemEventHandler {
static String topic(String operationType, String operationId, String level) {
return StringBuilderUtils
.buildString(operationType, operationId, level, (a, b, c, builder) -> {
// /sys-event/{operationType}/{operationId}/{level}
builder.append("/sys-event/")
.append(a)
.append('/')
.append(b)
.append('/')
.append(c);
});
}
void handle(SystemEvent event);
}

View File

@@ -0,0 +1,13 @@
package org.jetlinks.community.event;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.stereotype.Component;
@Component
public class SystemEventHandlerRegister {
public SystemEventHandlerRegister(ObjectProvider<SystemEventHandler> handlers){
handlers.forEach(SystemEventHolder::register);
}
}

View File

@@ -0,0 +1,54 @@
package org.jetlinks.community.event;
import lombok.extern.slf4j.Slf4j;
import org.jetlinks.community.Operation;
import reactor.core.Disposable;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
@Slf4j
public class SystemEventHolder {
private static final List<SystemEventHandler> eventHandlers = new CopyOnWriteArrayList<>();
public static Disposable register(SystemEventHandler handler) {
eventHandlers.add(handler);
return () -> eventHandlers.remove(handler);
}
public static void error(Operation operation, String code, Object detail) {
log.error("{} {} :{}", operation, code, detail);
if (eventHandlers.isEmpty()) {
return;
}
fireEvent(new SystemEvent(SystemEvent.Level.error, code, operation, detail));
}
public static void warn(Operation operation, String code, Object detail) {
log.warn("{} {} :{}", operation, code, detail);
if (eventHandlers.isEmpty()) {
return;
}
fireEvent(new SystemEvent(SystemEvent.Level.warn, code, operation, detail));
}
public static void info(Operation operation, String code, Object detail) {
log.info("{} {} :{}", operation, code, detail);
if (eventHandlers.isEmpty()) {
return;
}
fireEvent(new SystemEvent(SystemEvent.Level.info, code, operation, detail));
}
private static void fireEvent(SystemEvent event) {
for (SystemEventHandler eventHandler : eventHandlers) {
try {
eventHandler.handle(event);
} catch (Throwable e) {
log.warn("handle system log error", e);
}
}
}
}

View File

@@ -0,0 +1,296 @@
package org.jetlinks.community.lock;
import org.reactivestreams.Publisher;
import org.reactivestreams.Subscription;
import reactor.core.CoreSubscriber;
import reactor.core.Disposable;
import reactor.core.publisher.*;
import reactor.core.scheduler.Schedulers;
import reactor.util.context.Context;
import javax.annotation.Nonnull;
import javax.annotation.Nullable;
import java.time.Duration;
import java.util.Deque;
import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicIntegerFieldUpdater;
import java.util.concurrent.atomic.AtomicReferenceFieldUpdater;
import java.util.function.Consumer;
class DefaultReactiveLock implements ReactiveLock {
@SuppressWarnings("all")
static final AtomicReferenceFieldUpdater<DefaultReactiveLock, LockingSubscriber>
PENDING = AtomicReferenceFieldUpdater
.newUpdater(DefaultReactiveLock.class, LockingSubscriber.class, "pending");
final Deque<LockingSubscriber<?>> queue = new ConcurrentLinkedDeque<>();
volatile LockingSubscriber<?> pending;
@Override
public <T> Flux<T> lock(Flux<T> job) {
return new LockingFlux<>(this, job);
}
@Override
public <T> Flux<T> lock(Flux<T> flux, Duration timeout) {
return new LockingFlux<>(this, flux, timeout);
}
@Override
public <T> Flux<T> lock(Flux<T> flux, Duration timeout, Flux<? extends T> fallback) {
return new LockingFlux<>(this, flux, timeout, fallback);
}
@Override
public <T> Mono<T> lock(Mono<T> job) {
return new LockingMono<>(this, job);
}
@Override
public <T> Mono<T> lock(Mono<T> mono, Duration timeout) {
return new LockingMono<>(this, mono, timeout);
}
@Override
public <T> Mono<T> lock(Mono<T> mono, Duration timeout, Mono<? extends T> fallback) {
return new LockingMono<>(this, mono, timeout, fallback);
}
protected void drain() {
if (PENDING.get(this) != null) {
return;
}
LockingSubscriber<?> locking;
for (; ; ) {
locking = queue.pollFirst();
if (locking == null) {
return;
}
if (locking.isDisposed()) {
continue;
}
if (PENDING.compareAndSet(this, null, locking)) {
//使用单独的线程池来调度,防止参与锁太多导致栈溢出.
Schedulers.parallel().schedule(locking::subscribe);
} else {
queue.addLast(locking);
}
break;
}
}
<T> void registerSubscriber(CoreSubscriber<? super T> actual,
Consumer<CoreSubscriber<? super T>> subscribeCallback,
@Nullable Duration timeout,
@Nullable Publisher<? extends T> timeoutFallback) {
registerSubscriber(new LockingSubscriber<>(
this,
actual,
subscribeCallback,
timeout,
timeoutFallback));
}
void registerSubscriber(LockingSubscriber<?> subscriber) {
if (PENDING.compareAndSet(this, null, subscriber)) {
subscriber.subscribe();
return;
}
queue.addLast(subscriber);
drain();
}
static class LockingFlux<T> extends FluxOperator<T, T> {
private final DefaultReactiveLock main;
private Duration timeout;
private Publisher<? extends T> timeoutFallback;
protected LockingFlux(DefaultReactiveLock main, Flux<? extends T> source) {
super(source);
this.main = main;
}
protected LockingFlux(DefaultReactiveLock main, Flux<? extends T> source, Duration timeout) {
super(source);
this.main = main;
this.timeout = timeout;
}
protected LockingFlux(DefaultReactiveLock main, Flux<? extends T> source, Duration timeout, Flux<? extends T> timeoutFallback) {
super(source);
this.main = main;
this.timeout = timeout;
this.timeoutFallback = timeoutFallback;
}
@Override
public void subscribe(@Nonnull CoreSubscriber<? super T> actual) {
Consumer<CoreSubscriber<? super T>> subscribeCallback = source::subscribe;
main.registerSubscriber(actual, subscribeCallback, timeout, timeoutFallback);
}
}
static class LockingMono<T> extends MonoOperator<T, T> {
private final DefaultReactiveLock main;
private Duration timeout;
private Publisher<? extends T> fallback;
protected LockingMono(DefaultReactiveLock main, Mono<? extends T> source) {
super(source);
this.main = main;
}
protected LockingMono(DefaultReactiveLock main, Mono<? extends T> source, Duration timeout) {
super(source);
this.main = main;
this.timeout = timeout;
}
protected LockingMono(DefaultReactiveLock main, Mono<? extends T> source, Duration timeout, Mono<? extends T> fallback) {
super(source);
this.main = main;
this.timeout = timeout;
this.fallback = fallback;
}
@Override
public void subscribe(@Nonnull CoreSubscriber<? super T> actual) {
Consumer<CoreSubscriber<? super T>> subscribeCallback = source::subscribe;
main.registerSubscriber(actual, subscribeCallback, timeout, fallback);
}
}
static class LockingSubscriber<T> extends BaseSubscriber<T> {
protected final DefaultReactiveLock main;
protected final CoreSubscriber<? super T> actual;
private final Consumer<CoreSubscriber<? super T>> subscriber;
private Disposable timeoutTask;
private final Publisher<? extends T> timeoutFallback;
@SuppressWarnings("all")
protected static final AtomicIntegerFieldUpdater<LockingSubscriber> statusUpdater =
AtomicIntegerFieldUpdater.newUpdater(LockingSubscriber.class, "status");
private volatile int status;
//初始
private static final int INIT = 0;
//订阅备用流
private static final int SUB_TIMEOUT_FALLBACK = -1;
//订阅原本上游流
private static final int SUB_SOURCE = 1;
//流结束
private static final int UN_SUB = -2;
public LockingSubscriber(DefaultReactiveLock main,
CoreSubscriber<? super T> actual,
Consumer<CoreSubscriber<? super T>> subscriber,
@Nullable Duration timeout,
@Nullable Publisher<? extends T> timeoutFallback) {
this.actual = actual;
this.main = main;
this.subscriber = subscriber;
this.timeoutFallback = timeoutFallback;
if (timeout != null) {
this.timeoutTask = Schedulers
.parallel()
.schedule(this::onTimeout, timeout.toMillis(), TimeUnit.MILLISECONDS);
}
}
private void onTimeout() {
if (statusUpdater.compareAndSet(this, INIT, SUB_TIMEOUT_FALLBACK)) {
//不代理订阅,直接取消流及释放当前锁,以免并发时等待备用流释放锁
doComplete();
if (timeoutFallback != null) {
timeoutFallback.subscribe(actual);
} else {
this.onError(new TimeoutException("Lock timed out"));
}
}
}
protected void subscribe() {
if (statusUpdater.compareAndSet(this, INIT, SUB_SOURCE)) {
if (timeoutTask != null && !timeoutTask.isDisposed()) {
timeoutTask.dispose();
}
subscriber.accept(this);
}
}
protected void complete() {
if (statusUpdater.compareAndSet(this, INIT, UN_SUB) || statusUpdater.compareAndSet(this, SUB_SOURCE, UN_SUB)) {
if (timeoutTask != null && !timeoutTask.isDisposed()) {
timeoutTask.dispose();
}
doComplete();
}
}
protected void doComplete() {
//防止非hookFinally触发的结束
if (!this.isDisposed()) {
this.cancel();
}
if (PENDING.compareAndSet(main, this, null)) {
main.drain();
}
}
@Override
protected final void hookOnError(@Nonnull Throwable throwable) {
actual.onError(throwable);
}
@Override
protected final void hookOnNext(@Nonnull T value) {
actual.onNext(value);
}
@Override
protected final void hookOnSubscribe(@Nonnull Subscription subscription) {
actual.onSubscribe(this);
}
@Override
protected final void hookOnComplete() {
actual.onComplete();
}
@Override
protected final void hookOnCancel() {
super.hookOnCancel();
}
@Override
protected final void hookFinally(@Nonnull SignalType type) {
complete();
}
@Override
@Nonnull
public Context currentContext() {
return actual.currentContext();
}
}
}

View File

@@ -0,0 +1,20 @@
package org.jetlinks.community.lock;
import com.github.benmanes.caffeine.cache.Caffeine;
import java.time.Duration;
import java.util.Map;
class DefaultReactiveLockManager implements ReactiveLockManager {
private final Map<String, ReactiveLock> cache = Caffeine
.newBuilder()
.expireAfterAccess(Duration.ofMinutes(30))
.<String, ReactiveLock>build()
.asMap();
@Override
public ReactiveLock getLock(String name) {
return cache.computeIfAbsent(name, ignore -> new DefaultReactiveLock());
}
}

View File

@@ -0,0 +1,76 @@
package org.jetlinks.community.lock;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.time.Duration;
/**
* 响应式锁
*
* @author zhouhao
* @since 2.2
*/
public interface ReactiveLock {
/**
* 对Mono进行加锁,将等待之前的lock完成后再执行.
*
* @param mono 要加锁的Mono
* @param <T> T
* @return 加锁后的Mono
*/
<T> Mono<T> lock(Mono<T> mono);
/**
* 对Mono进行加锁,将等待之前的lock完成后再执行,若等待锁时间超过{@link Duration},则报错{@link java.util.concurrent.TimeoutException}
*
* @param mono 要加锁的Mono
* @param timeout 等待锁的时间
* @param <T> T
* @return 加锁后的Mono
*/
<T> Mono<T> lock(Mono<T> mono, Duration timeout);
/**
* 对Mono进行加锁,将等待之前的lock完成后再执行,若等待锁时间超过{@link Duration},则切换到回退流
*
* @param mono 要加锁的Mono
* @param timeout 等待锁的时间
* @param fallback 发生超时时要订阅的回退流
* @param <T> T
* @return 加锁后的Mono
*/
<T> Mono<T> lock(Mono<T> mono, Duration timeout, Mono<? extends T> fallback);
/**
* 对Flux进行加锁,将等待之前的lock完成后再执行.
*
* @param flux 要加锁的Flux
* @param <T> T
* @return 加锁后的Flux
*/
<T> Flux<T> lock(Flux<T> flux);
/**
* 对Flux进行加锁,将等待之前的lock完成后再执行,若等待锁时间超过{@link Duration},则报错{@link java.util.concurrent.TimeoutException}
*
* @param flux 要加锁的Flux
* @param timeout 等待锁的时间
* @param <T> T
* @return 加锁后的Mono
*/
<T> Flux<T> lock(Flux<T> flux, Duration timeout);
/**
* 对Flux进行加锁,将等待之前的lock完成后再执行,若等待锁时间超过{@link Duration},则切换到回退流
*
* @param flux 要加锁的Flux
* @param timeout 等待锁的时间
* @param fallback 发生超时时要订阅的回退流
* @param <T> T
* @return 加锁后的Mono
*/
<T> Flux<T> lock(Flux<T> flux, Duration timeout, Flux<? extends T> fallback);
}

View File

@@ -0,0 +1,39 @@
package org.jetlinks.community.lock;
/**
* 响应式锁持有器,用于通过静态方法获取锁.
* <pre>{@code
*
* Mono<MyEntity> execute(MyEntity entity){
*
* return ReactiveLockHolder
* .getLock("lock-test:"+entity.getId())
* .lock(updateAndGet(entity));
*
* }
* }</pre>
*
* @author zhouhao
* @see ReactiveLock
* @see ReactiveLockManager
* @since 2.2
*/
public class ReactiveLockHolder {
private static ReactiveLockManager lockManager = new DefaultReactiveLockManager();
static void setup(ReactiveLockManager manager) {
lockManager = manager;
}
/**
* 根据锁名称获取锁
*
* @param name 锁名称
* @return 锁
*/
public static ReactiveLock getLock(String name) {
return lockManager.getLock(name);
}
}

View File

@@ -0,0 +1,20 @@
package org.jetlinks.community.lock;
/**
* 响应式锁管理器
*
* @author zhouhao
* @see ReactiveLock
* @since 2.2
*/
public interface ReactiveLockManager {
/**
* 根据名称获取一个锁.
*
* @param name 锁名称
* @return 锁
*/
ReactiveLock getLock(String name);
}

View File

@@ -0,0 +1,35 @@
package org.jetlinks.community.reactorql.aggregation;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.SqlFragments;
import org.hswebframework.ezorm.rdb.operator.dml.FunctionColumn;
import org.jetlinks.community.spi.Provider;
import org.reactivestreams.Publisher;
import reactor.core.publisher.Mono;
import java.util.function.Function;
/**
* 聚合函数支持.
*
* @author zhangji 2025/1/22
* @since 2.3
*/
public interface AggregationSupport extends Function<Publisher<?>, Mono<?>> {
Provider<AggregationSupport> supports = Provider.create(AggregationSupport.class);
String getId();
String getName();
SqlFragments createSql(FunctionColumn column);
static AggregationSupport getNow(String id) {
return AggregationSupport.supports
.get(id.toUpperCase())
.orElseGet(() -> AggregationSupport.supports
.get(id.toLowerCase())
.orElseGet(() -> AggregationSupport.supports.getNow(id)));
}
}

View File

@@ -0,0 +1,82 @@
package org.jetlinks.community.reactorql.aggregation;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.BatchSqlFragments;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.SqlFragments;
import org.hswebframework.ezorm.rdb.operator.dml.FunctionColumn;
import org.jetlinks.reactor.ql.supports.agg.MapAggFeature;
import org.jetlinks.reactor.ql.utils.CastUtils;
import org.reactivestreams.Publisher;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.math.MathFlux;
import java.util.Comparator;
import java.util.function.Function;
import static org.jetlinks.reactor.ql.supports.DefaultReactorQLMetadata.addGlobal;
/**
*
* @author zhangji 2025/1/22
* @since 2.3
*/
@AllArgsConstructor
public enum InternalAggregationSupports implements AggregationSupport {
COUNT("总数", Flux::count, 0),
//去重计数
DISTINCT_COUNT("总数(去重)", flux -> flux.distinct().count(), 0) {
@Override
public SqlFragments createSql(FunctionColumn column) {
return new BatchSqlFragments().addSql("count(distinct ", column.getColumn(), ")");
}
},
MIN("最小值",
numberFlux -> MathFlux.min(numberFlux.map(CastUtils::castNumber), Comparator.comparing(Number::doubleValue)), null),
MAX("最大值", numberFlux -> MathFlux.max(numberFlux.map(CastUtils::castNumber), Comparator.comparing(Number::doubleValue)), null),
AVG("平均值", numberFlux -> MathFlux.averageDouble(numberFlux.map(CastUtils::castNumber), Number::doubleValue), null),
SUM("总和", numberFlux -> MathFlux.sumDouble(numberFlux.map(CastUtils::castNumber), Number::doubleValue), 0),
FIRST("第一个值", numberFlux -> numberFlux.take(1).singleOrEmpty(), null),
LAST("最后一个值", numberFlux -> numberFlux.takeLast(1).singleOrEmpty(), null),
// MEDIAN("中位数", numberFlux -> Mono.empty(), null),//中位数
// SPREAD("极差", numberFlux -> Mono.empty(), null),//差值
// STDDEV("标准差", numberFlux -> Mono.empty(), null),//标准差
;
static {
for (InternalAggregationSupports value : values()) {
addGlobal(new MapAggFeature(value.getId(), value::apply));
AggregationSupport.supports.register(value.getId(), value);
}
}
public static void register(){
}
@Getter
private final String name;
private final Function<Flux<?>, Mono<?>> computer;
@Getter
private final Object defaultValue;
@Override
public SqlFragments createSql(FunctionColumn column) {
return new BatchSqlFragments()
.addSql(name() + "(").addSql(column.getColumn()).addSql(")");
}
@Override
public String getId() {
return name();
}
@Override
public Mono<?> apply(Publisher<?> publisher) {
return computer.apply(Flux.from(publisher));
}
}

View File

@@ -0,0 +1,25 @@
package org.jetlinks.community.reactorql.function;
import lombok.Getter;
import lombok.Setter;
import org.jetlinks.core.metadata.DataType;
import org.jetlinks.core.metadata.PropertyMetadata;
import java.util.List;
/**
*
* @author zhangji 2025/1/22
* @since 2.3
*/
@Getter
@Setter
public class FunctionInfo {
private String id;
private String name;
private DataType outputType;
private List<PropertyMetadata> args;
}

View File

@@ -0,0 +1,72 @@
package org.jetlinks.community.reactorql.function;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.SqlFragments;
import org.jetlinks.core.metadata.DataType;
import org.jetlinks.community.spi.Provider;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
/**
* 函数支持,用于定义在可以在ReactorQL中使用的函数.
*
* @author zhangji 2025/1/22
* @since 2.3
*/
public interface FunctionSupport {
Provider<FunctionSupport> supports = Provider.create(FunctionSupport.class);
String getId();
String getName();
/**
* 是否支持列的数据类型
*
* @param type 数据类型
* @return 是否支持
*/
boolean isSupported(DataType type);
/**
* 获取输出数据类型
*
* @return 输出数据类型
*/
DataType getOutputType();
/**
* 创建SQL函数片段
*
* @param column 列名
* @param args 参数
* @return SQL函数片段
*/
SqlFragments createSql(String column, Map<String, Object> args);
/**
* 查找支持的函数
*
* @param type 数据类型
* @return 函数信息
*/
static List<FunctionInfo> lookup(DataType type) {
return supports
.getAll()
.stream()
.filter(support -> support.isSupported(type))
.map(FunctionSupport::toInfo)
.collect(Collectors.toList());
}
default FunctionInfo toInfo() {
FunctionInfo info = new FunctionInfo();
info.setId(getId());
info.setOutputType(getOutputType());
info.setName(getName());
return info;
}
}

View File

@@ -0,0 +1,70 @@
package org.jetlinks.community.reactorql.function;
import com.google.common.collect.Sets;
import lombok.AllArgsConstructor;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.SqlFragments;
import org.jetlinks.core.metadata.DataType;
import org.jetlinks.core.metadata.types.ArrayType;
import org.jetlinks.core.metadata.types.LongType;
import java.util.Collections;
import java.util.Map;
import java.util.Set;
/**
*
* @author zhangji 2025/1/22
* @since 2.3
*/
@AllArgsConstructor
public enum InternalFunctionSupport implements FunctionSupport {
array_len("集合长度", LongType.GLOBAL, ArrayType.ID);
private final String name;
private final DataType outputType;
private final Set<String> supportTypes;
static {
for (InternalFunctionSupport value : values()) {
InternalFunctionSupport.supports.register(value.getId(), value);
}
}
public static void register(){
}
InternalFunctionSupport(String name, DataType outputType, String... supportTypes) {
this.name = name;
this.outputType = outputType;
this.supportTypes = Collections.unmodifiableSet(
Sets.newHashSet(supportTypes)
);
}
@Override
public String getId() {
return name();
}
@Override
public String getName() {
return name;
}
@Override
public boolean isSupported(DataType type) {
return supportTypes.contains(type.getId());
}
@Override
public DataType getOutputType() {
return outputType;
}
@Override
public SqlFragments createSql(String column, Map<String, Object> args) {
return SqlFragments.of(getId() + "(", column, ")");
}
}

View File

@@ -0,0 +1,424 @@
package org.jetlinks.community.reactorql.impl;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.google.common.collect.Maps;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.Setter;
import net.sf.jsqlparser.expression.*;
import net.sf.jsqlparser.expression.operators.relational.ExpressionList;
import net.sf.jsqlparser.statement.select.Select;
import net.sf.jsqlparser.statement.select.SubSelect;
import org.apache.commons.collections4.CollectionUtils;
import org.hswebframework.ezorm.core.param.Term;
import org.hswebframework.ezorm.rdb.executor.SqlRequest;
import org.hswebframework.ezorm.rdb.executor.SqlRequests;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.BatchSqlFragments;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.EmptySqlFragments;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.SqlFragments;
import org.hswebframework.ezorm.rdb.operator.dml.FunctionColumn;
import org.hswebframework.ezorm.rdb.operator.dml.FunctionTerm;
import org.hswebframework.web.api.crud.entity.TermExpressionParser;
import org.hswebframework.web.bean.FastBeanCopier;
import org.jetlinks.community.reactorql.aggregation.AggregationSupport;
import org.jetlinks.community.utils.ConverterUtils;
import org.jetlinks.community.utils.ReactorUtils;
import org.jetlinks.core.metadata.Jsonable;
import org.jetlinks.reactor.ql.DefaultReactorQLContext;
import org.jetlinks.reactor.ql.ReactorQL;
import org.jetlinks.reactor.ql.ReactorQLMetadata;
import org.jetlinks.reactor.ql.ReactorQLRecord;
import org.jetlinks.reactor.ql.feature.FeatureId;
import org.jetlinks.reactor.ql.feature.ValueMapFeature;
import org.jetlinks.reactor.ql.supports.DefaultReactorQLMetadata;
import org.jetlinks.reactor.ql.utils.CastUtils;
import org.reactivestreams.Publisher;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiFunction;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.stream.Collectors;
/**
*
* @author zhangji 2025/1/22
* @since 2.3
*/
public class ComplexExistsFunction implements ValueMapFeature {
//集合中的元素
public static final String COL_ELEMENT = "_element";
//集合自身
public static final String COL_SELF = "_self";
//原始行
public static final String COL_ROW = "_row";
public static final ComplexExistsFunction INSTANCE = new ComplexExistsFunction();
public static final String function = "complex_exists";
static {
DefaultReactorQLMetadata.addGlobal(INSTANCE);
}
public static void register() {
}
public static ExistsSpec createExistsSpec(Object val) {
if (val instanceof ExistsSpec) {
return ((ExistsSpec) val);
}
if (val instanceof Map) {
ExistsSpec spec = new ExistsSpec();
spec.fromJson(new JSONObject((Map) val));
return spec;
}
if (val instanceof String) {
ExistsSpec spec = new ExistsSpec();
spec.setFilter(TermExpressionParser.parse(val.toString()));
return spec;
}
return FastBeanCopier.copy(val, new ExistsSpec());
}
private static ExistsProcessor createExprProcessor(Expression expr) {
String sql;
if (expr instanceof Select) {
sql = expr.toString();
} else if (expr instanceof SubSelect) {
sql = ((SubSelect) expr).getSelectBody().toString();
} else if (expr instanceof BinaryExpression) {
sql = "select 1 from t where " + expr;
} else {
throw new UnsupportedOperationException("不支持的表达式:" + expr);
}
SqlRequest request = SqlRequests.of(sql);
ReactorQL ql = ReactorQL
.builder()
.sql(request.getSql())
.build();
return new ReactorQLProcessor(request, ql);
}
private static ExistsProcessor createExprProcessor(String str) {
if (str.startsWith("select") || str.startsWith("SELECT")) {
SqlRequest request = SqlRequests.of(str);
ReactorQL ql = ReactorQL
.builder()
.sql(request.getSql())
.build();
return new ReactorQLProcessor(request, ql);
}
ExistsSpec spec = new ExistsSpec();
spec.setFilter(TermExpressionParser.parse(str));
return spec.compile();
}
@Override
public Function<ReactorQLRecord, Publisher<?>> createMapper(Expression expression, ReactorQLMetadata metadata) {
net.sf.jsqlparser.expression.Function func = ((net.sf.jsqlparser.expression.Function) expression);
ExpressionList args = func.getParameters();
if (args == null || args.getExpressions() == null || args.getExpressions().size() < 2) {
throw new UnsupportedOperationException("complex_exists函数参数错误");
}
Function<ReactorQLRecord, Mono<ExistsProcessor>> processorMapper;
List<Function<ReactorQLRecord, Publisher<?>>> mappers = args
.getExpressions()
.stream()
.skip(1)
.map(expr -> ValueMapFeature.createMapperNow(expr, metadata))
.collect(Collectors.toList());
Expression firstExpr = args.getExpressions().get(0);
//以字符串来定义 complex_exists('name is test')
if (firstExpr instanceof StringValue) {
Mono<ExistsProcessor> processor = Mono.just(createExprProcessor(((StringValue) firstExpr).getValue()));
processorMapper = record -> processor;
} else if (firstExpr instanceof NumericBind) {
int argIndex = ((NumericBind) firstExpr).getBindId() - 1;
processorMapper = record -> Mono.justOrEmpty(
record.getContext().getParameter(argIndex).map(this::transformProcessor)
);
} else if (firstExpr instanceof JdbcParameter) {
int argIndex = ((JdbcParameter) firstExpr).getIndex() - 1;
processorMapper = record -> Mono.justOrEmpty(
record.getContext().getParameter(argIndex).map(this::transformProcessor)
);
}
// complex_exists(select 1 from dual where ,arr)
else if (firstExpr instanceof JdbcNamedParameter) {
String name = ((JdbcNamedParameter) firstExpr).getName();
processorMapper = record -> Mono.justOrEmpty(
transformProcessor(record.getContext().getParameter(name))
);
} else {
Mono<ExistsProcessor> processor = Mono.just(createExprProcessor(firstExpr));
processorMapper = record -> processor;
}
return record ->
processorMapper
.apply(record)
.flatMap(processor -> {
if (mappers.size() == 1) {
return processor.apply(
record,
mappers.get(0).apply(record));
}
return processor.apply(
record,
Flux.fromIterable(mappers)
.flatMap(mapper -> mapper.apply(record))
);
});
}
@Override
public String getId() {
return FeatureId.ValueMap.of(function).getId();
}
private ExistsProcessor transformProcessor(Object value) {
if (value instanceof ExistsProcessor) {
return (ExistsProcessor) value;
}
if (value instanceof ExistsSpec) {
return ((ExistsSpec) value).compile();
}
return null;
}
public interface ExistsProcessor extends BiFunction<ReactorQLRecord, Publisher<?>, Mono<Boolean>> {
@Override
Mono<Boolean> apply(ReactorQLRecord record, Publisher<?> publisher);
Mono<Boolean> apply(ReactorQLRecord record, List<?> list);
}
@AllArgsConstructor
static class ReactorQLProcessor implements ExistsProcessor {
private final SqlRequest request;
private final ReactorQL ql;
public Mono<Boolean> apply(ReactorQLRecord record, List<?> list) {
Flux<?> data = Flux
.fromIterable(list)
.map(v -> {
Map<String, Object> _data = Maps.newHashMapWithExpectedSize(3);
_data.put(COL_ROW,record.getRecord());
_data.put(COL_ELEMENT, v);
_data.put(COL_SELF, list);
return _data;
});
DefaultReactorQLContext ctx = new DefaultReactorQLContext((t) -> data);
for (Object parameter : request.getParameters()) {
ctx.bind(parameter);
}
return ql.start(ctx)
.hasElements();
}
@Override
public Mono<Boolean> apply(ReactorQLRecord record, Publisher<?> publisher) {
if (publisher instanceof Mono) {
return ((Mono<?>) publisher)
.map(ConverterUtils::convertToList)
.flatMap(list -> apply(record, list));
}
return Flux
.from(publisher)
.as(CastUtils::flatStream)
.collectList()
.flatMap(list -> apply(record, list));
}
@Override
public String toString() {
return request.toNativeSql();
}
}
private static FunctionTerm convertFunctionTerm(Object obj) {
if (obj instanceof FunctionTerm) {
return (FunctionTerm) obj;
}
if (obj instanceof Map) {
return convertFunctionTerm(new JSONObject((Map) obj));
}
throw new UnsupportedOperationException("不支持的类型:" + obj);
}
private static FunctionTerm convertFunctionTerm(JSONObject obj) {
FunctionTerm term = new FunctionTerm();
FastBeanCopier.copy(obj, term, "terms");
JSONArray terms = obj.getJSONArray("terms");
if (terms != null) {
terms
.forEach(o -> {
term.addTerm(convertFunctionTerm((JSONObject) o));
});
}
return term;
}
@Getter
@Setter
public static class ExistsSpec implements Jsonable {
//过滤
private List<Term> filter;
//聚合
private List<FunctionTerm> aggregation;
@Override
public void fromJson(JSONObject json) {
FastBeanCopier.copy(json, this, "aggregation");
JSONArray aggregation = json.getJSONArray("aggregation");
if (aggregation != null) {
this.aggregation = aggregation
.stream()
.map(ComplexExistsFunction::convertFunctionTerm)
.collect(Collectors.toList());
}
}
//todo 其他简便配置的方式?如: 任意满足,全部满足等
public void walkTerms(Consumer<Term> consumer) {
if (filter != null) {
filter.forEach(consumer);
}
if (aggregation != null) {
aggregation.forEach(consumer);
}
}
private void applyAggregation(BatchSqlFragments fragments,
AtomicInteger count,
FunctionTerm agg,
Map<String, String> distinct,
Map<Term, String> aliasMapping) {
// 大小写都支持
AggregationSupport support = AggregationSupport.getNow(agg.getFunction());
FunctionColumn col = new FunctionColumn();
col.setFunction(agg.getFunction());
col.setColumn("this['" + agg.getColumn() + "']");
col.setOpts(agg.getOpts() == null ? null : Maps.transformValues(agg.getOpts(), Object.class::cast));
SqlFragments frg = support.createSql(col);
String sqlStr = frg.toRequest().toNativeSql();
String alias = distinct.get(frg.toRequest().toNativeSql());
if (alias == null) {
alias = "_agg_" + count.incrementAndGet();
distinct.put(sqlStr, alias);
aliasMapping.put(agg, alias);
if (count.get() > 1) {
fragments.addSql(",");
}
fragments.add(frg).addSql(alias);
}
if (CollectionUtils.isNotEmpty(agg.getTerms())) {
for (Term term : agg.getTerms()) {
if (term instanceof FunctionTerm) {
applyAggregation(fragments,
count,
((FunctionTerm) term),
distinct,
aliasMapping);
}
}
}
}
private List<Term> createHavingTerm(List<? extends Term> aggregation,
Map<Term, String> aliasMapping) {
List<Term> terms = new ArrayList<>(aggregation.size());
for (Term functionTerm : aggregation) {
Term term = new Term();
term.setColumn(aliasMapping.getOrDefault(functionTerm, functionTerm.getColumn()));
term.setTermType(functionTerm.getTermType());
term.setValue(functionTerm.getValue());
term.setOptions(functionTerm.getOptions());
term.setType(Term.Type.and);
if (CollectionUtils.isNotEmpty(functionTerm.getTerms())) {
term.setTerms(createHavingTerm(functionTerm.getTerms(), aliasMapping));
}
terms.add(term);
}
return terms;
}
public ExistsProcessor compile() {
SqlFragments cols;
SqlFragments having;
if (CollectionUtils.isNotEmpty(aggregation)) {
Map<String, String> distinct = new HashMap<>();
Map<Term, String> aliasMapping = new HashMap<>();
BatchSqlFragments cols_ = new BatchSqlFragments();
AtomicInteger count = new AtomicInteger(1);
count.incrementAndGet();
cols_.addSql(COL_SELF,",",COL_ELEMENT);
for (FunctionTerm functionTerm : aggregation) {
applyAggregation(cols_, count, functionTerm, distinct, aliasMapping);
}
cols = cols_;
List<Term> havingTerms = createHavingTerm(aggregation, aliasMapping);
having = ReactorUtils.createFilterSql(havingTerms);
} else {
cols = SqlFragments.ONE;
having = EmptySqlFragments.INSTANCE;
}
SqlFragments where = ReactorUtils.createFilterSql(filter);
BatchSqlFragments fragments = new BatchSqlFragments();
if (having.isNotEmpty()) {
fragments.addSql("select 1 from (");
}
fragments.addSql("select").addFragments(cols).addSql("from t");
if (where.isNotEmpty()) {
fragments.addSql("where").addFragments(where);
}
if (having.isNotEmpty()) {
fragments.add(SqlFragments.RIGHT_BRACKET)
.add(SqlFragments.WHERE)
.addFragments(having);
}
SqlRequest request = fragments.toRequest();
ReactorQL ql = ReactorQL.builder()
.sql(request.getSql())
.build();
return new ReactorQLProcessor(request, ql);
}
}
}

View File

@@ -9,65 +9,158 @@ import org.hswebframework.ezorm.rdb.operator.builder.fragments.SqlFragments;
import org.hswebframework.web.i18n.LocaleUtils;
import org.jetlinks.core.metadata.DataType;
import org.jetlinks.core.metadata.types.*;
import org.jetlinks.community.utils.ConverterUtils;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import static org.jetlinks.community.reactorql.term.TermType.OPTIONS_NATIVE_SQL;
import java.util.stream.Collectors;
import java.util.stream.Stream;
@Getter
public enum FixedTermTypeSupport implements TermTypeSupport {
eq("等于", "eq"),
neq("不等于", "neq"),
eq("等于", "eq") {
@Override
public boolean isSupported(DataType type) {
return !type.getType().equals(ArrayType.ID) && super.isSupported(type);
}
},
neq("不等于", "neq") {
@Override
public boolean isSupported(DataType type) {
return !type.getType().equals(ArrayType.ID) && super.isSupported(type);
}
gt("大于", "gt", DateTimeType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID),
gte("大于等于", "gte", DateTimeType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID),
lt("小于", "lt", DateTimeType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID),
lte("小于等于", "lte", DateTimeType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID),
},
notnull("不为空", "notnull", false) {
@Override
protected String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s%s", property, getName());
}
},
isnull("为空", "isnull", false) {
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s%s", property, getName());
}
},
btw("在...之间", "btw", DateTimeType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID) {
gt("大于", "gt", DateTimeType.ID, ShortType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID),
gte("大于等于", "gte", DateTimeType.ID, ShortType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID),
lt("小于", "lt", DateTimeType.ID, ShortType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID),
lte("小于等于", "lte", DateTimeType.ID, ShortType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID),
btw("在...之间", "btw", DateTimeType.ID, ShortType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID) {
@Override
protected Object convertValue(Object val, Term term) {
return val;
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s在%s之间", property, arrayToSpec(expect));
}
},
nbtw("不在...之间", "nbtw", DateTimeType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID) {
nbtw("不在...之间", "nbtw", DateTimeType.ID, ShortType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID) {
@Override
protected Object convertValue(Object val, Term term) {
return val;
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s不在%s之间", property, createValueDesc(expect));
}
},
in("在...之中", "in", StringType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID, EnumType.ID) {
in("在...之中", "in", StringType.ID, ShortType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID, EnumType.ID) {
@Override
protected Object convertValue(Object val, Term term) {
return val;
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s在%s之中", property, createValueDesc(expect));
}
},
nin("不在...之中", "nin", StringType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID, EnumType.ID) {
nin("不在...之中", "nin", StringType.ID, ShortType.ID, IntType.ID, LongType.ID, FloatType.ID, DoubleType.ID, EnumType.ID) {
@Override
protected Object convertValue(Object val, Term term) {
return val;
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s不在%s之中", property, createValueDesc(expect));
}
},
contains_all("全部包含在...之中", "contains_all", ArrayType.ID) {
@Override
protected Object convertValue(Object val, Term term) {
return val;
return ConverterUtils.convertToList(val);
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s全部包含在%s之中", property, createValueDesc(expect));
}
},
contains_any("任意包含在...之中", "contains_any", ArrayType.ID) {
@Override
protected Object convertValue(Object val, Term term) {
return val;
return ConverterUtils.convertToList(val);
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s任意包含在%s之中", property, createValueDesc(expect));
}
},
not_contains("不包含在...之中", "not_contains", ArrayType.ID) {
@Override
protected Object convertValue(Object val, Term term) {
return val;
return ConverterUtils.convertToList(val);
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s不包含在%s之中", property, createValueDesc(expect));
}
},
@@ -88,7 +181,7 @@ public enum FixedTermTypeSupport implements TermTypeSupport {
nlike("不包含字符", "str_nlike", StringType.ID) {
@Override
protected Object convertValue(Object val, Term term) {
return like.convertValue(val,term);
return like.convertValue(val, term);
}
},
@@ -98,16 +191,37 @@ public enum FixedTermTypeSupport implements TermTypeSupport {
protected void appendFunction(String column, PrepareSqlFragments fragments) {
fragments.addSql("gt(math.divi(math.sub(now(),", column, "),1000),");
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s距离当前时间大于%s秒", property, expect);
}
},
time_lt_now("距离当前时间小于...秒", "time_lt_now", DateTimeType.ID) {
@Override
protected void appendFunction(String column, PrepareSqlFragments fragments) {
fragments.addSql("lt(math.divi(math.sub(now(),", column, "),1000),");
}
@Override
protected String createValueDesc(Object expect) {
return arrayToSpec(expect);
}
@Override
public String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s距离当前时间小于%s秒", property, expect);
}
};
private final String text;
private final boolean needValue;
private final Set<String> supportTypes;
private final String function;
@@ -119,6 +233,13 @@ public enum FixedTermTypeSupport implements TermTypeSupport {
this.supportTypes = Sets.newHashSet(supportTypes);
}
FixedTermTypeSupport(String text, String function, boolean needValue, String... supportTypes) {
this.text = text;
this.function = function;
this.needValue = needValue;
this.supportTypes = Sets.newHashSet(supportTypes);
}
@Override
public boolean isSupported(DataType type) {
return supportTypes.isEmpty() || supportTypes.contains(type.getType());
@@ -135,7 +256,11 @@ public enum FixedTermTypeSupport implements TermTypeSupport {
}
protected void appendFunction(String column, PrepareSqlFragments fragments) {
fragments.addSql(function + "(", column, ",");
if (needValue) {
fragments.addSql(function + "(", column, ",");
} else {
fragments.addSql(function + "(", column, ")");
}
}
@Override
@@ -158,6 +283,20 @@ public enum FixedTermTypeSupport implements TermTypeSupport {
return fragments;
}
static String arrayToSpec(Object value) {
if (value == null) {
return "[]";
}
List<String> list = ConverterUtils
.convertToList(value, String::valueOf);
if (list.size() > 8) {
return Stream
.concat(list.stream().limit(8), Stream.of("..."))
.collect(Collectors.joining(",", "[", "]"));
}
return list.toString();
}
@Override
public String getType() {
return name();
@@ -167,4 +306,26 @@ public enum FixedTermTypeSupport implements TermTypeSupport {
public String getName() {
return LocaleUtils.resolveMessage("message.term_type_" + name(), text);
}
protected String createValueDesc(Object expect) {
return String.valueOf(expect);
}
protected String createDefaultDesc(String property, Object expect, Object actual) {
return String.format("%s%s(%s)", property, getName(), expect);
}
@Override
public String createDesc(String property, Object expect, Object actual) {
//在国际化资源文件中查找对应的描述
// {0}=属性名称,{1}=期望值
//如: message.term_type_neq_desc={0}不等于{1}
return LocaleUtils.resolveMessage(
"message.term_type_" + name() + "_desc",
createDefaultDesc(property, expect, expect),
property,
createValueDesc(expect),
actual
);
}
}

View File

@@ -3,13 +3,23 @@ package org.jetlinks.community.reactorql.term;
import lombok.SneakyThrows;
import org.hswebframework.ezorm.core.param.Term;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.SqlFragments;
import org.hswebframework.web.i18n.LocaleUtils;
import org.jetlinks.community.utils.ReactorUtils;
import org.jetlinks.core.metadata.DataType;
import reactor.core.publisher.Mono;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.function.BiFunction;
/**
* 查询条件类型支持
*
* @author zhouhao
* @see org.jetlinks.community.utils.ReactorUtils#createFilter(List)
* @since 2.0
*/
public interface TermTypeSupport {
/**
@@ -40,6 +50,18 @@ public interface TermTypeSupport {
*/
SqlFragments createSql(String column, Object value, Term term);
/**
* 重构条件
*
* @param tableName 表名
* @param term 条件
* @param refactor 重构函数
* @return 重构后的条件
*/
default Term refactorTerm(String tableName, Term term, BiFunction<String,Term,Term> refactor){
return refactor.apply(tableName,term);
}
/**
* 判断是否已经过时,过时的条件应当不可选择.
*
@@ -58,8 +80,21 @@ public interface TermTypeSupport {
return TermType.of(getType(), getName());
}
default String createDesc(String property, Object expect, Object actual) {
return String.format("%s%s(%s)", property, getName(), expect);
return LocaleUtils.resolveMessage(
"message.term_" + getType() + "_desc",
String.format("%s%s(%s)", property, getName(), expect),
property,
getName(),
expect,
actual
);
}
default String createActualDesc(String property, Object actual) {
return property + " = " + actual;
}
/**

View File

@@ -35,6 +35,9 @@ public class TermTypes {
}
public static Optional<TermTypeSupport> lookupSupport(String type) {
if (type == null) {
return Optional.empty();
}
return Optional.ofNullable(supports.get(type));
}
}

View File

@@ -0,0 +1,120 @@
package org.jetlinks.community.reactorql.term;
import org.hswebframework.ezorm.core.param.Term;
import org.hswebframework.ezorm.rdb.executor.SqlRequest;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.NativeSql;
import org.jetlinks.community.reactorql.function.FunctionSupport;
import org.springframework.util.StringUtils;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.BiFunction;
import java.util.function.Function;
import java.util.stream.Collectors;
/**
* @author zhangji 2025/2/8
* @since 2.3
*/
public class TermUtils {
public static List<Term> expandTermToList(List<Term> terms) {
Map<String, List<Term>> termsMap = expandTerm(terms);
List<Term> termList = new ArrayList<>();
for (List<Term> values : termsMap.values()) {
termList.addAll(values);
}
return termList;
}
public static Map<String, List<Term>> expandTerm(List<Term> terms) {
Map<String, List<Term>> termCache = new LinkedHashMap<>();
expandTerm(terms, termCache);
return termCache;
}
private static void expandTerm(List<Term> terms, Map<String, List<Term>> container) {
if (terms == null) {
return;
}
for (Term term : terms) {
if (StringUtils.hasText(term.getColumn())) {
List<Term> termList = container.get(term.getColumn());
if (termList == null) {
List<Term> list = new ArrayList<>();
list.add(term);
container.put(term.getColumn(), list);
} else {
termList.add(term);
container.put(term.getColumn(), termList);
}
}
if (term.getTerms() != null) {
expandTerm(term.getTerms(), container);
}
}
}
public static Term refactorTerm(String tableName,
Term term,
BiFunction<String, String, String> columnRefactor) {
if (term.getColumn() == null) {
return term;
}
String[] arr = term.getColumn().split("[.]");
List<TermValue> values = TermValue.of(term);
if (values.isEmpty()) {
return term;
}
Function<TermValue, Object> parser = value -> {
//上游变量
if (value.getSource() == TermValue.Source.variable
|| value.getSource() == TermValue.Source.upper) {
term.getOptions().add(TermType.OPTIONS_NATIVE_SQL);
return columnRefactor.apply(tableName, value.getValue().toString());
}
//指标
else if (value.getSource() == TermValue.Source.metric) {
term.getOptions().add(TermType.OPTIONS_NATIVE_SQL);
return tableName + "['" + arr[1] + "_metric_" + value.getMetric() + "']";
}
//函数, 如: array_len() , device_prop()
else if (value.getSource() == TermValue.Source.function) {
SqlRequest request = FunctionSupport
.supports
.getNow(value.getFunction())
.createSql(columnRefactor.apply(tableName, value.getColumn()), value.getArgs())
.toRequest();
return NativeSql.of(request.getSql(), request.getParameters());
}
//手动设置值
else {
return value.getValue();
}
};
Object val;
if (values.size() == 1) {
val = parser.apply(values.get(0));
} else {
val = values
.stream()
.map(parser)
.collect(Collectors.toList());
}
if (term.getOptions().contains(TermType.OPTIONS_NATIVE_SQL) && !(val instanceof NativeSql)) {
val = NativeSql.of(String.valueOf(val));
}
term.setColumn(columnRefactor.apply(tableName, term.getColumn()));
term.setValue(val);
return term;
}
}

View File

@@ -0,0 +1,105 @@
package org.jetlinks.community.reactorql.term;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Getter;
import lombok.Setter;
import org.hswebframework.ezorm.core.param.Term;
import org.hswebframework.web.bean.FastBeanCopier;
import java.io.Serializable;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
/**
*
* @author zhangji 2025/2/27
* @since 2.3
*/
@Getter
@Setter
public class TermValue implements Serializable {
private static final long serialVersionUID = 1;
@Schema(description = "来源")
private Source source;
@Schema(description = "[source]为[manual]时不能为空")
private Object value;
@Schema(description = "[source]为[metric]时不能为空")
private String metric;
@Schema(description = "[source]为[function]时不能为空")
private String function;
@Schema(description = "[source]为[function]时有效")
private String column;
@Schema(description = "[source]为[function]时有效")
private Map<String, Object> args;
public static TermValue manual(Object value) {
TermValue termValue = new TermValue();
termValue.setValue(value);
termValue.setSource(Source.manual);
return termValue;
}
public static TermValue metric(String metric) {
TermValue termValue = new TermValue();
termValue.setMetric(metric);
termValue.setSource(Source.metric);
return termValue;
}
public static List<TermValue> of(Term term) {
return of(term.getValue());
}
public static List<TermValue> of(Object value) {
if (value == null) {
return Collections.emptyList();
}
if (value instanceof Map) {
return Collections.singletonList(FastBeanCopier.copy(value, new TermValue()));
}
if (value instanceof TermValue) {
return Collections.singletonList(((TermValue) value));
}
if (value instanceof Collection) {
return ((Collection<?>) value)
.stream()
.flatMap(val -> of(val).stream())
.collect(Collectors.toList());
}
return Collections.singletonList(TermValue.manual(value));
}
public enum Source {
/**
* 和manual一样,
* 兼容{@link org.jetlinks.pro.relation.utils.VariableSource.Source#fixed}
*/
fixed,
manual,
metric,
variable,
/**
* 和variable一样,兼容{@link org.jetlinks.pro.relation.utils.VariableSource.Source#upper}
*/
upper,
/**
* 函数
*
* @see org.jetlinks.pro.reactorql.function.FunctionSupport
*/
function
}
}

View File

@@ -20,6 +20,8 @@ public interface DataReferenceManager {
//数据类型: 设备接入网关
String TYPE_DEVICE_GATEWAY = "device-gateway";
//数据类型: 产品
String TYPE_PRODUCT = "product";
//数据类型: 网络组件
String TYPE_NETWORK = "network";
//数据类型:关系配置
@@ -53,7 +55,6 @@ public interface DataReferenceManager {
*/
Flux<DataReferenceInfo> getReferences(String dataType);
/**
* 断言数据没有被引用,如果存在引用,则抛出异常: {@link DataReferencedException}
*
@@ -69,4 +70,18 @@ public interface DataReferenceManager {
.flatMap(list -> Mono.error(new DataReferencedException(dataType, dataId, list)));
}
/**
* 断言数据没有被引用,如果存在引用,则抛出异常: {@link DataReferencedException}
*
* @param dataType 数据类型
* @param dataId 数据ID
* @return void
*/
default Mono<Void> assertNotReferenced(String dataType, String dataId, String code, Object... args) {
return this
.getReferences(dataType, dataId)
.collectList()
.filter(CollectionUtils::isNotEmpty)
.flatMap(list -> Mono.error(new DataReferencedException(dataType, dataId, list, code, args)));
}
}

View File

@@ -27,4 +27,17 @@ public class DataReferencedException extends I18nSupportException {
super.setI18nCode("error.data.referenced");
}
public DataReferencedException(String dataType,
String dataId,
List<DataReferenceInfo> referenceList,
String code,
Object... args) {
this.dataType = dataType;
this.dataId = dataId;
this.referenceList = referenceList;
super.setI18nCode(code);
super.setArgs(args);
}
}

View File

@@ -0,0 +1,14 @@
package org.jetlinks.community.resource.ui;
import lombok.extern.slf4j.Slf4j;
import org.jetlinks.community.resource.ClassPathJsonResourceProvider;
@Slf4j
public class UiMenuResourceProvider extends ClassPathJsonResourceProvider {
public static final String TYPE = "ui-menus";
public UiMenuResourceProvider() {
super(TYPE, "classpath*:/ui/*/baseMenu.json");
}
}

View File

@@ -0,0 +1,83 @@
package org.jetlinks.community.resource.ui;
import lombok.Getter;
import lombok.Setter;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.jetlinks.community.resource.Resource;
import org.jetlinks.community.resource.ResourceProvider;
import org.jetlinks.community.resource.SimpleResource;
import org.jetlinks.community.utils.ObjectMappers;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternResolver;
import org.springframework.util.StreamUtils;
import reactor.core.publisher.Flux;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
@Slf4j
public class UiResourceProvider implements ResourceProvider {
public static final String TYPE = "ui";
private static final ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
private List<Resource> cache;
@Override
public String getType() {
return TYPE;
}
@Override
public Flux<Resource> getResources() {
return Flux.fromIterable(cache == null ? cache = read() : cache);
}
@SneakyThrows
private List<Resource> read() {
List<Resource> resources = new ArrayList<>();
try {
for (org.springframework.core.io.Resource resource : resolver.getResources("classpath*:/ui/*/package.json")) {
try (InputStream stream = resource.getInputStream()) {
String s = StreamUtils.copyToString(stream, StandardCharsets.UTF_8);
Module m = ObjectMappers.parseJson(s, Module.class);
String path = resource.getURL().getPath();
String[] parts = path.split("/");
if (parts.length > 2) {
m.setPath(parts[parts.length - 3] + "/" + parts[parts.length - 2]);
resources.add(m.toResource());
}
}
}
} catch (Throwable e) {
log.warn("load ui resource error", e);
}
return resources;
}
@Override
public Flux<Resource> getResources(Collection<String> id) {
return Flux.empty();
}
@Getter
@Setter
public static class Module {
private String id;
private String name;
private String description;
private String path;
public SimpleResource toResource() {
id = StringUtils.isBlank(id) ? name : id;
return SimpleResource.of(id, TYPE, ObjectMappers.toJsonString(this));
}
}
}

View File

@@ -101,9 +101,13 @@ public class I18nSpec implements Serializable {
private static List<I18nSpec> of(Object... args) {
List<I18nSpec> codes = new ArrayList<>();
for (Object arg : args) {
I18nSpec i18NSpec = new I18nSpec();
i18NSpec.setDefaultMessage(arg);
codes.add(i18NSpec);
if (arg instanceof I18nSpec) {
codes.add((I18nSpec)arg);
} else {
I18nSpec i18NSpec = new I18nSpec();
i18NSpec.setDefaultMessage(arg);
codes.add(i18NSpec);
}
}
return codes;
}

View File

@@ -0,0 +1,133 @@
package org.jetlinks.community.terms;
import lombok.AllArgsConstructor;
import org.hswebframework.ezorm.core.param.Term;
import org.hswebframework.ezorm.rdb.metadata.RDBColumnMetadata;
import org.hswebframework.ezorm.rdb.metadata.RDBTableMetadata;
import org.hswebframework.ezorm.rdb.metadata.TableOrViewMetadata;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.*;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.term.AbstractTermFragmentBuilder;
import org.jetlinks.community.utils.ConverterUtils;
import java.util.List;
/**
* 关联子查询动态条件抽象类,统一封装和另外的表进行关联查询的动态条件.
*
* <pre>{@code
*
* // 动态参数
* {
* "column":"subId",
* "value":"name is 123" //支持的格式见: ConverterUtils.convertTerms
* }
*
* exists(
* select 1 from {SubTableName} _st where _st.{subTableColumn} = t.sub_id
* and _st.name = ?
* )
*
* }</pre>
*
* @author zhouhao
* @see ConverterUtils#convertTerms(Object)
* @since 2.2
*/
public abstract class SubTableTermFragmentBuilder extends AbstractTermFragmentBuilder {
public SubTableTermFragmentBuilder(String termType, String name) {
super(termType, name);
}
static final SqlFragments AND_L = SqlFragments.single("and (");
@Override
public SqlFragments createFragments(String columnFullName, RDBColumnMetadata column, Term term) {
List<Term> terms = ConverterUtils.convertTerms(term.getValue());
String subTableName = getSubTableName();
RDBTableMetadata subTable = column
.getOwner()
.getSchema()
.getTable(subTableName, false)
.orElseThrow(() -> new UnsupportedOperationException("unsupported " + getSubTableName()));
RDBColumnMetadata subTableColumn = subTable.getColumnNow(getSubTableColumn());
BatchSqlFragments sqlFragments = new BatchSqlFragments(5, 0);
if (term.getOptions().contains("not")) {
sqlFragments.add(SqlFragments.NOT);
}
sqlFragments
.addSql("exists(select 1 from", subTable.getFullName(), getTableAlias(),
"where", subTableColumn.getFullName(getTableAlias()), "=", columnFullName);
SqlFragments where = builder.createTermFragments(subTable, terms);
if (where.isNotEmpty()) {
sqlFragments
.add(AND_L)
.addFragments(where)
.add(SqlFragments.RIGHT_BRACKET);
}
sqlFragments.add(SqlFragments.RIGHT_BRACKET);
return sqlFragments;
}
/**
* @return 子表名
*/
protected abstract String getSubTableName();
/**
* @return 子表列名
*/
protected String getSubTableColumn() {
return "id";
}
/**
* @return 子查询别名
*/
protected String getTableAlias() {
return "_st";
}
// 动态条件构造器
TermsBuilder builder = new TermsBuilder(getTableAlias());
@AllArgsConstructor
static class TermsBuilder extends AbstractTermsFragmentBuilder<TableOrViewMetadata> {
private final String tableAlias;
@Override
protected SqlFragments createTermFragments(TableOrViewMetadata parameter,
List<Term> terms) {
return super.createTermFragments(parameter, terms);
}
@Override
protected SqlFragments createTermFragments(TableOrViewMetadata table,
Term term) {
if (term.getValue() instanceof NativeSql) {
NativeSql sql = ((NativeSql) term.getValue());
return SimpleSqlFragments.of(sql.getSql(), sql.getParameters());
}
RDBColumnMetadata column = table.getColumn(term.getColumn()).orElse(null);
if (column == null) {
return EmptySqlFragments.INSTANCE;
}
TermFragmentBuilder builder = column
.findFeature(TermFragmentBuilder.createFeatureId(term.getTermType()))
.orElse(null);
if (builder != null) {
return builder
.createFragments(column.getFullName(tableAlias), column, term);
}
return EmptySqlFragments.INSTANCE;
}
}
}

View File

@@ -11,10 +11,10 @@ import org.hswebframework.ezorm.core.param.TermType;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.NativeSql;
import org.hswebframework.web.bean.FastBeanCopier;
import org.hswebframework.web.i18n.LocaleUtils;
import org.jetlinks.community.reactorql.term.TermTypeSupport;
import org.jetlinks.community.reactorql.term.TermTypes;
import org.jetlinks.core.metadata.Jsonable;
import org.jetlinks.core.utils.SerializeUtils;
import org.jetlinks.community.reactorql.term.TermTypeSupport;
import org.jetlinks.community.reactorql.term.TermTypes;
import org.jetlinks.reactor.ql.supports.DefaultPropertyFeature;
import org.springframework.util.StringUtils;
@@ -67,6 +67,12 @@ public class TermSpec implements Jsonable, Serializable {
@Schema(description = "是否为物模型变量")
private boolean metadata;
@Schema(description = "触发条件描述")
private I18nSpec triggerSpec;
@Schema(description = "实际触发描述")
private I18nSpec actualSpec;
@Override
public JSONObject toJson() {
@SuppressWarnings("all")
@@ -100,11 +106,36 @@ public class TermSpec implements Jsonable, Serializable {
.ifPresent(support -> termSpec.matched = support.matchBlocking(termSpec.getExpected(),termSpec.getActual()));
}
if (termSpec.matched != null && termSpec.matched) {
actualDesc.add(termSpec.getDisplayName() + " = " + termSpec.getActual());
actualDesc.add(
termSpec.getActualSpec() != null
? termSpec.getActualSpec().resolveI18nMessage()
: TermTypes
.lookupSupport(termSpec.getTermType())
.map(support -> support.createActualDesc(termSpec.getDisplayName(), termSpec.getActual()))
.orElse(termSpec.getDisplayName() + " = " + termSpec.getActual())
);
}
if (CollectionUtils.isNotEmpty(termSpec.children)) {
boolean conditionMatch = true;
Set<String> andConditionDesc = new HashSet<>();
for (TermSpec child : termSpec.children) {
actualDesc.addAll(parseTermSpecActualDesc(child));
Set<String> desc = parseTermSpecActualDesc(child);
// 任一条件不满足则不添加and条件的描述
if (child.getColumn() != null && CollectionUtils.isEmpty(desc)) {
conditionMatch = false;
}
if (child.getType() == Term.Type.or) {
actualDesc.addAll(desc);
}
if (child.getType() == Term.Type.and) {
andConditionDesc.addAll(desc);
}
}
// 所有条件都满足时再添加and条件匹配的描述
if (conditionMatch) {
actualDesc.addAll(andConditionDesc);
}
}
return actualDesc;
@@ -115,6 +146,15 @@ public class TermSpec implements Jsonable, Serializable {
});
}
public static TermSpec ofTermSpecs(List<TermSpec> termSpecs) {
TermSpec spec = new TermSpec();
if (CollectionUtils.isEmpty(termSpecs)) {
return spec;
}
spec.setChildren(new ArrayList<>(termSpecs));
return spec;
}
public static List<TermSpec> of(List<Term> terms, BiConsumer<Term, TermSpec> customizer) {
if (terms == null) {
return null;
@@ -156,7 +196,7 @@ public class TermSpec implements Jsonable, Serializable {
this.actual = DefaultPropertyFeature
.GLOBAL
.getProperty(this.column, context)
.orElse(null);
.orElse(this.actual);
if (expectIsExpr) {
this.expected = DefaultPropertyFeature
.GLOBAL
@@ -164,10 +204,10 @@ public class TermSpec implements Jsonable, Serializable {
.orElse(null);
expectIsExpr = false;
}
TermTypes
.lookupSupport(getTermType())
.ifPresent(support -> this.matched = support.matchBlocking(expected, actual));
}
if (this.children != null) {
for (TermSpec child : this.children) {
@@ -222,7 +262,11 @@ public class TermSpec implements Jsonable, Serializable {
.orElse(null);
if (support != null) {
hasSpec = true;
builder.append(support.createDesc(getDisplayName(), expected, actual));
builder.append(
triggerSpec != null
? triggerSpec.resolveI18nMessage()
: support.createDesc(getDisplayName(), expected, actual)
);
}
}
List<TermSpec> children = compressChildren();
@@ -240,13 +284,27 @@ public class TermSpec implements Jsonable, Serializable {
builder.append('!');
}
}
builder.append("( ");
toString(builder, children);
builder.append(" )");
if (StringUtils.hasText(builder)) {
builder.append("( ");
toString(builder, children);
builder.append(" )");
} else {
toString(builder, children);
}
}
}
public void compress() {
List<TermSpec> children = compressChildren();
if (children != this.children) {
setChildren(children);
}
for (TermSpec child : children) {
child.compress();
}
}
public void setActual(Object actual) {
this.actual = actual;

View File

@@ -1,7 +1,17 @@
package org.jetlinks.community.topic;
import lombok.Generated;
import org.jetlinks.core.lang.SeparatedCharSequence;
import org.jetlinks.core.lang.SharedPathString;
import org.jetlinks.core.utils.StringBuilderUtils;
import org.jetlinks.community.utils.TopicUtils;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.Consumer;
import java.util.stream.Collectors;
public interface Topics {
@@ -17,6 +27,13 @@ public interface Topics {
return StringBuilderUtils.buildString(creatorId, topic, Topics::creator);
}
static SeparatedCharSequence creator(String creatorId, SeparatedCharSequence topic) {
// /user/{creatorId}/{topic}
return SharedPathString
.of(new String[]{"", "user", creatorId})
.append(topic);
}
static void creator(String creatorId, String topic, StringBuilder builder) {
builder.append("/user/").append(creatorId);
if (topic.charAt(0) != '/') {
@@ -25,6 +42,101 @@ public interface Topics {
builder.append(topic);
}
@Deprecated
static String tenant(String tenantId, String topic) {
if (!topic.startsWith("/")) {
topic = "/" + topic;
}
return String.join("", "/tenant/", tenantId, topic);
}
@Deprecated
static List<String> tenants(List<String> tenants, String topic) {
List<String> topics = new ArrayList<>(tenants.size());
for (String tenant : tenants) {
topics.add(tenant(tenant, topic));
}
return topics;
}
@Deprecated
static String deviceGroup(String groupId, String topic) {
if (!topic.startsWith("/")) {
topic = "/" + topic;
}
return String.join("", "/device-group/", groupId, topic);
}
@Deprecated
static List<String> deviceGroups(List<String> groupIds, String topic) {
List<String> topics = new ArrayList<>(groupIds.size());
for (String groupId : groupIds) {
topics.add(deviceGroup(groupId, topic));
}
return topics;
}
/**
* 根据绑定信息构造topic
*
* @param bindings 绑定信息
* @param topic topic
* @return topic
*/
static List<String> bindings(List<Map<String, Object>> bindings, String topic) {
List<String> topics = new ArrayList<>(bindings.size());
bindings(bindings, topic, topics::add);
return topics;
}
static void bindings(List<Map<String, Object>> bindings,
String topic,
Consumer<String> consumer) {
for (Map<String, Object> binding : bindings) {
consumer.accept(binding(String.valueOf(binding.get("type")), String.valueOf(binding.get("id")), topic));
}
}
static void bindings(List<Map<String, Object>> bindings,
SeparatedCharSequence topic,
Consumer<SeparatedCharSequence> consumer) {
for (Map<String, Object> binding : bindings) {
consumer.accept(binding(String.valueOf(binding.get("type")), String.valueOf(binding.get("id")), topic));
}
}
static void relations(List<Map<String, Object>> relations,
String topic,
Consumer<String> consumer) {
for (Map<String, Object> relation : relations) {
consumer.accept(
relation(String.valueOf(relation.get("type")),
String.valueOf(relation.get("id")),
String.valueOf(relation.get("rel")),
topic)
);
}
}
static void relations(List<Map<String, Object>> relations,
SeparatedCharSequence topic,
Consumer<SeparatedCharSequence> consumer) {
for (Map<String, Object> relation : relations) {
consumer.accept(
relation(String.valueOf(relation.get("type")),
String.valueOf(relation.get("id")),
String.valueOf(relation.get("rel")),
topic)
);
}
}
static void binding(String type, String id, String topic, StringBuilder builder) {
builder.append('/')
.append(type)
@@ -63,6 +175,39 @@ public interface Topics {
builder.append(topic);
}
static String relation(String objectType, String objectId, String relation, String topic) {
return StringBuilderUtils.buildString(objectType, objectId, relation, topic, Topics::relation);
}
static SeparatedCharSequence relation(String objectType, String objectId, String relation, SeparatedCharSequence topic) {
return SharedPathString.of(new String[]{"", objectType, objectId, relation}).append(topic);
}
static String binding(String type, String id, String topic) {
return StringBuilderUtils.buildString(type, id, topic, Topics::binding);
}
static SeparatedCharSequence binding(String type, String id, SeparatedCharSequence topic) {
return SharedPathString.of(new String[]{"", type, id}).append(topic);
}
@Deprecated
static String tenantMember(String memberId, String topic) {
if (!topic.startsWith("/")) {
topic = "/" + topic;
}
return String.join("", "/member/", memberId, topic);
}
@Deprecated
static List<String> tenantMembers(List<String> members, String topic) {
return members
.stream()
.map(id -> tenantMember(id, topic))
.collect(Collectors.toList());
}
String allDeviceRegisterEvent = "/_sys/registry-device/*/register";
String allDeviceUnRegisterEvent = "/_sys/registry-device/*/unregister";
String allDeviceMetadataChangedEvent = "/_sys/registry-device/*/metadata";
@@ -110,11 +255,34 @@ public interface Topics {
return "/_sys/registry-product/" + deviceId + "/" + event;
}
/**
* 重构topic,将topic拼接上租户等信息的前缀
*
* @param topic topic
* @param configs 包含的信息
* @return
*/
@SuppressWarnings("all")
static Set<String> refactorTopic(String topic, Map<String, Object> configs) {
return TopicUtils.refactorTopic(configs, topic);
}
static String alarm(String targetType, String targetId, String alarmId) {
// /alarm/{targetType}/{targetId}/{alarmId}/record
return String.join("", "/alarm/", targetType, "/", targetId, "/", alarmId, "/record");
}
static String alarmHandleHistory(String targetType, String targetId, String alarmId) {
// /alarm/{targetType}/{targetId}/{alarmId}/handle-history
return String.join("", "/alarm/", targetType, "/", targetId, "/", alarmId, "/handle-history");
}
static String alarmLog(String targetType, String targetId, String alarmId, String recordId) {
// /alarm/{targetType}/{targetId}/{alarmId}/{recordId}/log
return String.join("", "/alarm/", targetType, "/", targetId, "/", alarmId, "/", recordId, "/log");
}
static String alarmRelieve(String targetType, String targetId, String alarmId) {
// /alarm/{targetType}/{targetId}/{alarmId}/relieve
return String.join("", "/alarm/", targetType, "/", targetId, "/", alarmId, "/relieve");

View File

@@ -1,23 +1,33 @@
package org.jetlinks.community.utils;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONArray;
import io.micrometer.core.instrument.Tags;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufUtil;
import io.netty.buffer.Unpooled;
import io.netty.util.CharsetUtil;
import lombok.SneakyThrows;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.collections4.ComparatorUtils;
import org.apache.commons.collections4.MapUtils;
import org.hswebframework.ezorm.core.param.Sort;
import org.hswebframework.ezorm.core.param.Term;
import org.hswebframework.web.api.crud.entity.TermExpressionParser;
import org.hswebframework.web.bean.FastBeanCopier;
import org.jetlinks.core.utils.StringBuilderUtils;
import org.jetlinks.reactor.ql.utils.CompareUtils;
import org.springframework.util.ConcurrentReferenceHashMap;
import org.springframework.util.StringUtils;
import reactor.core.publisher.Flux;
import java.nio.charset.Charset;
import java.nio.charset.CharsetEncoder;
import java.util.*;
import java.util.function.BiFunction;
import java.util.function.Function;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
public class ConverterUtils {
private static final Pattern HEX_PATTERN = Pattern.compile("^[0-9a-fA-F]+$");
/**
* 尝试转换值为集合,如果不是集合格式则直接返回该值
*
@@ -28,7 +38,7 @@ public class ConverterUtils {
public static Object tryConvertToList(Object value, Function<Object, Object> converter) {
List<Object> list = convertToList(value, converter);
if (list.size() == 1) {
return list.get(0);
return converter.apply(list.get(0));
}
return list;
}
@@ -92,15 +102,45 @@ public class ConverterUtils {
* @return 排序后的流
*/
public static <T> Flux<T> convertSortedStream(Flux<T> flux, Collection<Sort> sorts) {
return convertSortedStream(flux, FastBeanCopier::getProperty, sorts);
}
/**
* 根据排序参数对指定对Flux流进行排序
*
* @param flux Flux
* @param sorts 排序参数
* @param propertyGetter 对比字段获取器,用于获取元素中的字段数据.
* @param <R> 流中元素类型
* @return 排序后的流
*/
public static <R> Flux<R> convertSortedStream(Flux<R> flux,
BiFunction<R, String, Object> propertyGetter,
Collection<Sort> sorts) {
if (CollectionUtils.isEmpty(sorts)) {
return flux;
}
List<Comparator<T>> comparators = new ArrayList<>(sorts.size());
return flux.sort(convertComparator(sorts, propertyGetter));
}
/**
* 将排序参数转为Comparator
*
* @param sorts 排序参数
* @param <R> 元素类型
* @return Comparator
*/
public static <R> Comparator<R> convertComparator(Collection<Sort> sorts,
BiFunction<R, String, Object> propertyGetter) {
if (CollectionUtils.isEmpty(sorts)) {
return Comparator.comparing(k -> 0);
}
List<Comparator<R>> comparators = new ArrayList<>(sorts.size());
for (Sort sort : sorts) {
String column = sort.getName();
Comparator<T> comparator = (left, right) -> {
Object leftVal = FastBeanCopier.copy(left, new HashMap<>()).get(column);
Object rightVal = FastBeanCopier.copy(right, new HashMap<>()).get(column);
Comparator<R> comparator = (left, right) -> {
Object leftVal = propertyGetter.apply(left, column);
Object rightVal = propertyGetter.apply(right, column);
return CompareUtils.compare(leftVal, rightVal);
};
if (sort.getOrder().equalsIgnoreCase("desc")) {
@@ -109,7 +149,18 @@ public class ConverterUtils {
comparators.add(comparator);
}
return flux.sort(ComparatorUtils.chainedComparator(comparators));
return ComparatorUtils.chainedComparator(comparators);
}
private static final String[] EMPTY_TAG = new String[0];
private static final Map<Map<String, Object>, Tags> tagCache = new ConcurrentReferenceHashMap<>();
public static Tags convertMapToTagsInfo(Map<String, Object> map) {
if (MapUtils.isEmpty(map)) {
return Tags.empty();
}
return tagCache.computeIfAbsent(map, m -> Tags.of(convertMapToTags(m)));
}
/**
@@ -121,29 +172,9 @@ public class ConverterUtils {
* @param map map
* @return tags
*/
@SneakyThrows
public static String[] convertMapToTags(Map<String, Object> map) {
if (MapUtils.isEmpty(map)) {
return new String[0];
}
String[] tags = new String[map.size() * 2];
int index = 0;
for (Map.Entry<String, Object> entry : map.entrySet()) {
String key = entry.getKey();
Object value = entry.getValue();
if (value == null) {
continue;
}
String strValue = value instanceof String
? String.valueOf(value)
: JSON.toJSONString(value);
tags[index++] = key;
tags[index++] = strValue;
}
if (tags.length > index) {
return Arrays.copyOf(tags, index);
}
return tags;
return org.jetlinks.sdk.server.utils.ConverterUtils.convertMapToTags(map);
}
/**
@@ -152,27 +183,146 @@ public class ConverterUtils {
* //name = xxx and age > 10
* convertTerms("name is xxx and age gt 10")
*
* convertTerms({"name":"xxx","age$gt":10})
* </pre>
*
* @param value
* @return 条件集合
*/
@SuppressWarnings("all")
@SneakyThrows
public static List<Term> convertTerms(Object value) {
if (value instanceof String) {
String strVal = String.valueOf(value);
//json字符串
if (strVal.startsWith("[")) {
value = JSON.parseArray(strVal);
return org.jetlinks.sdk.server.utils.ConverterUtils.convertTerms(value);
}
public static String byteBufToString(ByteBuf buf,
Charset charset) {
return StringBuilderUtils
.buildString(
buf, charset,
(_buf, _charset, builder) -> byteBufToString(builder, _buf, _charset));
}
/**
* 将字符串转为ByteBuf,字符串中包含\x开头的内容将按16进制解析为byte.
*
* @param str 字符串
* @param charset 字符集
*/
public static ByteBuf stringToByteBuf(CharSequence str, Charset charset) {
ByteBuf buf = Unpooled.buffer(str.length());
stringToByteBuf(str, buf, charset);
return buf;
}
/**
* 将字符串转为ByteBuf,字符串中包含\x开头的内容将按16进制解析为byte.
*
* @param str 字符串
* @param buf ByteBuf
* @param charset 字符集
*/
public static void stringToByteBuf(CharSequence str,
ByteBuf buf,
Charset charset) {
int idx = 0;
int len = str.length();
while (idx < len) {
char c = str.charAt(idx);
if (c == '\\' && str.charAt(idx + 1) == 'x') {
buf.writeByte(ByteBufUtil.decodeHexByte(str, idx + 2));
idx += 4;
} else {
//表达式
return TermExpressionParser.parse(strVal);
buf.writeCharSequence(String.valueOf(c), charset);
idx++;
}
}
if (value instanceof List) {
return new JSONArray(((List) value)).toJavaList(Term.class);
} else {
throw new UnsupportedOperationException("unsupported term value:" + value);
}
private static final Set<Character>
visibleChar = new HashSet<>(
Arrays.asList(
' ', '!', '"', '#', '$', '%', '&', '\'', '(', ')', '*', '+', ',', '-', '.', '/',
'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '<', '=', '>', '?',
'@', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
'[', '\\', ']', '^', '_', '`',
'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
'{', '|', '}', '~', '`', '[', ']', '{', '}', ';', '\'', ',', '.', '/', '?', '<', '>', '~', '!', '@', '#', '$', '%', '^', '&', '*', '(', ')', '-', '_', '+', '=', '|',
'·', '', '@', '#', '¥', '%', '…', '&', '*', '', '', '—', '+', '|', '', '“', '”', '《', '》', '', '、',
'。', '', '', '', '', '【', '】', '、', '·', '', '·', '', '@', '#', '¥', '%', '…', '&', '*', '', '',
'—', '+', '|', '', '“', '”'
)
);
/**
* 将netty的ByteBuf转为字符串,buf中包含非指定字符集的字符,则转换为16进制如: \x00
*
* @param builder StringBuilder 用于接收字符串
* @param buf ByteBuf
* @param charset 字符集
*/
public static void byteBufToString(StringBuilder builder,
ByteBuf buf,
Charset charset) {
int len = buf.readableBytes();
int idx = buf.readerIndex();
CharsetEncoder encoder = CharsetUtil.encoder(charset);
int avgPerChar = (int) encoder.averageBytesPerChar();
int maxPerChar = (int) encoder.maxBytesPerChar();
while (len > 0) {
if (len >= avgPerChar && ByteBufUtil.isText(buf, idx, avgPerChar, charset)) {
CharSequence cs = buf.getCharSequence(idx, avgPerChar, charset);
int clen = cs.length();
for (int i = 0; i < clen; i++) {
char c = cs.charAt(i);
if (visibleChar.contains(c)) {
builder.append(c);
} else {
builder
.append("\\x")
.append(ByteBufUtil.hexDump(buf, idx + i, 1));
}
}
len -= avgPerChar;
idx += avgPerChar;
continue;
}
if (len >= maxPerChar && ByteBufUtil.isText(buf, idx, maxPerChar, charset)) {
CharSequence cs = buf.getCharSequence(idx, maxPerChar, charset);
builder.append(cs);
len -= maxPerChar;
idx += maxPerChar;
continue;
}
//不可识别的转为hex
builder
.append("\\x")
.append(ByteBufUtil.hexDump(buf, idx, 1));
len--;
idx++;
}
}
public static ByteBuf convertBuffer(Object obj) {
return org.jetlinks.sdk.server.utils.ConverterUtils.convertNettyBuffer(obj);
}
public static boolean isHexEncoded(String str) {
return StringUtils.hasText(str) &&
str.length() % 2 == 0 &&
HEX_PATTERN.matcher(str).matches();
}
}

View File

@@ -0,0 +1,44 @@
package org.jetlinks.community.utils;
/**
* @author gyl
* @since 2.0
*/
public class FormatUtils {
private static final String[] formats = {"B", "KB", "MB", "GB"};
/**
* 单位为字节的大小转换为最大单位
*
* @param bytes
* @return
*/
public static String formatDataSize(long bytes) {
int i = 0;
float total = bytes;
while (total >= 1024 && i < formats.length - 1) {
total /= 1024;
i++;
}
return String.format("%.2f%s", total, formats[i]);
}
/**
* 将毫秒格式化为x小时x分钟x秒表示
*
* @param diffTime
* @return
*/
public static String calculateLifeTime(long diffTime) {
long hours = diffTime / 3600000;
long minutes = (diffTime % 3600000) / 60000;
long seconds = (diffTime % 60000) / 1000;
return hours + "小时" + minutes + "分钟" + seconds + "";
}
}

View File

@@ -3,6 +3,7 @@ package org.jetlinks.community.utils;
import org.hswebframework.ezorm.core.param.Term;
import org.hswebframework.ezorm.rdb.executor.SqlRequest;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.AbstractTermsFragmentBuilder;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.EmptySqlFragments;
import org.hswebframework.ezorm.rdb.operator.builder.fragments.SqlFragments;
import org.hswebframework.web.bean.FastBeanCopier;
import org.jetlinks.community.reactorql.term.FixedTermTypeSupport;
@@ -132,6 +133,13 @@ public class ReactorUtils {
return createFilter(terms, converter, (arg, data) -> arg);
}
public static SqlFragments createFilterSql(List<Term> terms) {
if (CollectionUtils.isEmpty(terms)) {
return EmptySqlFragments.INSTANCE;
}
return termBuilder.createTermFragments(null, terms);
}
@SuppressWarnings("all")
public static <T> Function<T, Mono<Boolean>> createFilter(List<Term> terms,
Function<T, Map<String, Object>> converter,

View File

@@ -0,0 +1,344 @@
package org.jetlinks.community.utils;
import com.google.common.collect.Sets;
import lombok.SneakyThrows;
import org.apache.commons.collections4.MapUtils;
import org.jetlinks.community.PropertyConstants;
import org.jetlinks.community.topic.Topics;
import org.jetlinks.core.lang.SeparatedCharSequence;
import org.jetlinks.core.lang.SharedPathString;
import org.jetlinks.core.message.*;
import org.jetlinks.core.message.collector.ReportCollectorDataMessage;
import org.jetlinks.core.message.event.ThingEventMessage;
import org.jetlinks.core.message.module.ThingModuleMessage;
import org.jetlinks.core.utils.StringBuilderUtils;
import org.springframework.util.StringUtils;
import reactor.function.Consumer3;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.function.BiConsumer;
import java.util.function.BiFunction;
import java.util.function.Consumer;
public class TopicUtils {
public static final List<Consumer3<Map<String, Object>, String, Consumer<String>>> TOPIC_REFACTOR_HOOK
= new CopyOnWriteArrayList<>();
public static Set<String> refactorTopic(Map<String, Object> configs, String original) {
if (MapUtils.isEmpty(configs)) {
return Collections.singleton(original);
}
Set<String> topics = Sets.newHashSetWithExpectedSize(2);
//原始的topic
topics.add(original);
//创建人
String creatorId = PropertyConstants.getFromMapOrElse(PropertyConstants.creatorId, configs, () -> null);
if (StringUtils.hasText(creatorId)) {
topics.add(org.jetlinks.community.topic.Topics.creator(creatorId, original));
}
// 执行hooks
if (!TOPIC_REFACTOR_HOOK.isEmpty()) {
for (Consumer3<Map<String, Object>, String, Consumer<String>> hook : TOPIC_REFACTOR_HOOK) {
hook.accept(configs, original, topics::add);
}
}
return topics;
}
public static Set<SeparatedCharSequence> refactorTopic(DeviceMessage message, SeparatedCharSequence original) {
return refactorTopic(message.getHeaders(), original);
}
public static Set<SeparatedCharSequence> refactorTopic(Map<String, Object> configs, SeparatedCharSequence original) {
if (MapUtils.isEmpty(configs)) {
return Collections.singleton(original);
}
Set<SeparatedCharSequence> container = Sets.newHashSetWithExpectedSize(2);
container.add(original);
//创建人
String creatorId = PropertyConstants.getFromMapOrElse(PropertyConstants.creatorId, configs, () -> null);
if (StringUtils.hasText(creatorId)) {
container.add(
Topics.creator(creatorId, original)
);
}
// 执行hooks
if (!TOPIC_REFACTOR_HOOK.isEmpty()) {
for (Consumer3<Map<String, Object>, String, Consumer<String>> hook : TOPIC_REFACTOR_HOOK) {
hook.accept(configs, original.toString(), (str) -> container.add(SharedPathString.of(str)));
}
}
return container;
}
public static Set<String> refactorTopic(ThingMessage message, String original) {
return refactorTopic(message.getHeaders(), original);
}
private static final TopicBuilder[] TOPIC_BUILDERS;
static {
TOPIC_BUILDERS = new TopicBuilder[MessageType.values().length];
{
SharedPathString shared = SharedPathString.of("/message/event");
//事件
createFastBuilder(MessageType.EVENT,
(message, builder) -> {
ThingEventMessage event = ((ThingEventMessage) message);
builder.append("/message/event/").append(event.getEvent());
},
(message, charSequences) -> {
ThingEventMessage event = ((ThingEventMessage) message);
return charSequences.isEmpty()
? shared.append(event.getEvent())
: charSequences.append(shared).append(event.getEvent());
});
}
//上报属性
createFastBuilder(MessageType.REPORT_PROPERTY, "/message/property/report");
//读取属性
createFastBuilder(MessageType.READ_PROPERTY, "/message/send/property/read");
//读取属性回复
createFastBuilder(MessageType.READ_PROPERTY_REPLY, "/message/property/read/reply");
//修改属性
createFastBuilder(MessageType.WRITE_PROPERTY, "/message/send/property/write");
//修改属性回复
createFastBuilder(MessageType.WRITE_PROPERTY_REPLY, "/message/property/write/reply");
//调用功能
createFastBuilder(MessageType.INVOKE_FUNCTION, "/message/send/function");
//调用功能回复
createFastBuilder(MessageType.INVOKE_FUNCTION_REPLY, "/message/function/reply");
//注册
createFastBuilder(MessageType.REGISTER, "/register");
//注销
createFastBuilder(MessageType.UN_REGISTER, "/unregister");
//拉取固件
createFastBuilder(MessageType.REQUEST_FIRMWARE, "/firmware/pull");
//拉取固件回复
createFastBuilder(MessageType.REQUEST_FIRMWARE_REPLY, "/firmware/pull/reply");
//上报固件信息
createFastBuilder(MessageType.REPORT_FIRMWARE, "/firmware/report");
//上报固件安装进度
createFastBuilder(MessageType.UPGRADE_FIRMWARE_PROGRESS, "/firmware/progress");
//推送固件
createFastBuilder(MessageType.UPGRADE_FIRMWARE, "/firmware/push");
//推送固件回复
createFastBuilder(MessageType.UPGRADE_FIRMWARE_REPLY, "/firmware/push/reply");
//未知
createFastBuilder(MessageType.UNKNOWN, "/message/unknown");
//应答消息 since 1.8
createFastBuilder(MessageType.ACKNOWLEDGE, "/message/acknowledge");
//日志
createFastBuilder(MessageType.LOG, "/message/log");
//透传
createFastBuilder(MessageType.DIRECT, "/message/direct");
//更新标签
createFastBuilder(MessageType.UPDATE_TAG, "/message/tags/update");
//上线
createFastBuilder(MessageType.ONLINE, "/online");
//离线
createFastBuilder(MessageType.OFFLINE, "/offline");
//断开连接
createFastBuilder(MessageType.DISCONNECT, "/disconnect");
//断开连接回复
createFastBuilder(MessageType.DISCONNECT_REPLY, "/disconnect/reply");
{
SharedPathString shared = SharedPathString.of("/message/children");
//子设备消息
createFastBuilder(MessageType.CHILD, (message, builder) -> {
Message msg = ((ChildDeviceMessage) message).getChildDeviceMessage();
if (msg instanceof ThingMessage) {
builder.append("/message/children/")
.append(((ThingMessage) msg).getThingId());
} else {
builder.append("/message/children");
}
appendMessageTopic(msg, builder);
}, (message, builder) -> {
Message msg = ((ChildDeviceMessage) message).getChildDeviceMessage();
if (msg instanceof ThingMessage) {
String thingId = ((ThingMessage) msg).getThingId();
builder = builder.isEmpty() ? shared.append(thingId) : builder.append(shared).append(thingId);
} else {
builder = builder.isEmpty() ? shared : builder.append(shared);
}
return createMessageTopic(msg, builder);
});
}
{
SharedPathString shared = SharedPathString.of("/message/children/reply");
//子设备消息回复
createFastBuilder(MessageType.CHILD_REPLY,
(message, builder) -> {
Message msg = ((ChildDeviceMessageReply) message).getChildDeviceMessage();
if (msg instanceof ThingMessage) {
builder.append("/message/children/reply/")
.append(((ThingMessage) msg).getThingId());
} else {
builder.append("/message/children/reply");
}
appendMessageTopic(msg, builder);
},
(message, builder) -> {
Message msg = ((ChildDeviceMessageReply) message).getChildDeviceMessage();
if (msg instanceof ThingMessage) {
String thingId = ((ThingMessage) msg).getThingId();
builder = builder.isEmpty() ? shared.append(thingId) : builder.append(shared).append(thingId);
} else {
builder = builder.isEmpty() ? shared : builder.append(shared);
}
return createMessageTopic(msg, builder);
});
}
//上报了新的物模型
createFastBuilder(MessageType.DERIVED_METADATA, "/metadata/derived");
//状态检查
createFastBuilder(MessageType.STATE_CHECK, "/message/state_check");
createFastBuilder(MessageType.STATE_CHECK_REPLY, "/message/state_check_reply");
{
SharedPathString shared = SharedPathString.of("/message/collector/report");
//数采相关消息 since 2.1
createFastBuilder(
MessageType.REPORT_COLLECTOR,
((message, stringBuilder) -> {
String addr = message.getHeaderOrElse(ReportCollectorDataMessage.ADDRESS, null);
stringBuilder.append("/message/collector/report");
if (StringUtils.hasText(addr)) {
if (!addr.startsWith("/")) {
stringBuilder.append('/');
}
stringBuilder.append(addr);
}
}),
(message, charSequences) -> {
String addr = message.getHeaderOrElse(ReportCollectorDataMessage.ADDRESS, null);
charSequences = charSequences.append(shared);
if (StringUtils.hasText(addr)) {
charSequences = charSequences.append(SharedPathString.of(addr));
}
return charSequences;
});
}
createFastBuilder(MessageType.READ_COLLECTOR_DATA, "/message/collector/read");
createFastBuilder(MessageType.READ_COLLECTOR_DATA_REPLY, "/message/collector/read/reply");
createFastBuilder(MessageType.WRITE_COLLECTOR_DATA, "/message/collector/write");
createFastBuilder(MessageType.WRITE_COLLECTOR_DATA_REPLY, "/message/collector/write/reply");
//模块消息 since 2.3
createFastBuilder(MessageType.MODULE,
(message, builder) -> {
ThingModuleMessage msg = ((ThingModuleMessage) message);
builder.append("/module/").append(msg.getModule());
appendMessageTopic(msg.getMessage(), builder);
},
(message, builder) -> {
ThingModuleMessage msg = ((ThingModuleMessage) message);
if (builder.isEmpty()) {
builder = SharedPathString.of(new String[]{
"", "module", msg.getModule()
});
} else {
builder = builder
.append("module")
.append(SharedPathString.of(msg.getModule()));
}
return createMessageTopic(msg.getMessage(), builder);
});
}
private static void createFastBuilder(MessageType messageType,
String topic) {
SharedPathString shared = SharedPathString.of(topic);
TOPIC_BUILDERS[messageType.ordinal()] = new TopicBuilder() {
@Override
public void build(Message message, StringBuilder builder) {
builder.append(topic);
}
@Override
public SeparatedCharSequence build(Message message, SeparatedCharSequence prefix) {
return prefix.isEmpty() ? shared : prefix.append(shared);
}
};
}
private static void createFastBuilder(MessageType messageType,
BiConsumer<Message, StringBuilder> builderBiConsumer,
BiFunction<Message, SeparatedCharSequence, SeparatedCharSequence> builderBiConsumer2) {
TOPIC_BUILDERS[messageType.ordinal()] = new TopicBuilder() {
@Override
public void build(Message message, StringBuilder builder) {
builderBiConsumer.accept(message, builder);
}
@Override
public SeparatedCharSequence build(Message message, SeparatedCharSequence prefix) {
return builderBiConsumer2.apply(message, prefix);
}
};
}
public static String createMessageTopic(Message message, String prefix) {
return StringBuilderUtils
.buildString(message, prefix, (msg, _prefix, builder) -> {
builder.append(_prefix);
TopicUtils.appendMessageTopic(msg, builder);
});
}
@SneakyThrows
public static SeparatedCharSequence createMessageTopic(Message message,
SeparatedCharSequence prefix) {
//根据消息类型枚举快速获取拼接器器
TopicBuilder fastBuilder = TOPIC_BUILDERS[message.getMessageType().ordinal()];
if (null != fastBuilder) {
//执行拼接
return fastBuilder.build(message, prefix);
} else {
//不支持的类型,则直接拼接类型
return prefix.append("message", message.getMessageType().name().toLowerCase());
}
}
@SneakyThrows
public static void appendMessageTopic(Message message, StringBuilder builder) {
//根据消息类型枚举快速获取拼接器器
TopicBuilder fastBuilder = TOPIC_BUILDERS[message.getMessageType().ordinal()];
if (null != fastBuilder) {
//执行拼接
fastBuilder.build(message, builder);
} else {
//不支持的类型,则直接拼接类型
builder.append("/message/").append(message.getMessageType().name().toLowerCase());
}
}
private interface TopicBuilder {
void build(Message message, StringBuilder builder);
default SeparatedCharSequence build(Message message, SeparatedCharSequence prefix) {
return prefix.append(
SharedPathString.of(StringBuilderUtils.buildString(message, this::build))
);
}
}
}

View File

@@ -0,0 +1,63 @@
package org.jetlinks.community.web;
import io.swagger.v3.oas.annotations.Hidden;
import lombok.AllArgsConstructor;
import lombok.SneakyThrows;
import org.hswebframework.web.authorization.annotation.Authorize;
import org.jetlinks.core.command.CommandSupport;
import org.jetlinks.core.metadata.FunctionMetadata;
import org.jetlinks.community.command.CommandSupportManagerProvider;
import org.jetlinks.community.command.CommandSupportManagerProviders;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
/**
* 获取平台内部命令信息接口
*
* @author zhouhao
* @since 2.2
*/
@RestController
@RequestMapping("/command-supports")
@Hidden
@AllArgsConstructor
public class CommandInfoController {
@GetMapping("/services")
@SneakyThrows
@Authorize
public Flux<CommandSupportManagerProvider.CommandSupportInfo> getServices() {
return Flux
.fromIterable(CommandSupportManagerProvider.supports.getAll())
.flatMap(provider -> provider
.getSupportInfo()
.map(s -> s.copy().appendService(provider.getProvider()))
.defaultIfEmpty(
CommandSupportManagerProvider
.CommandSupportInfo
.of(provider.getProvider(), null, null)));
}
@GetMapping("/service/{serviceId}/commands")
@SneakyThrows
@Authorize
public Flux<FunctionMetadata> getServiceCommands(@PathVariable String serviceId) {
return CommandSupportManagerProviders
.getCommandSupport(serviceId)
.flatMapMany(CommandSupport::getCommandMetadata);
}
@GetMapping("/service/{serviceId}/exists")
@SneakyThrows
@Authorize
public Mono<Boolean> getServiceCommandSupport(@PathVariable String serviceId) {
return CommandSupportManagerProviders
.getCommandSupport(serviceId)
.hasElement()
.onErrorResume(err -> Mono.just(false));
}
}

View File

@@ -9,6 +9,7 @@ import org.hswebframework.web.authorization.exception.UnAuthorizedException;
import org.jetlinks.community.resource.Resource;
import org.jetlinks.community.resource.ResourceManager;
import org.jetlinks.community.resource.TypeScriptDeclareResourceProvider;
import org.jetlinks.community.resource.ui.UiResourceProvider;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
@@ -17,6 +18,7 @@ import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.util.Collections;
import java.util.HashMap;
@RestController
@RequestMapping("/system/resources")
@@ -26,6 +28,15 @@ public class SystemResourcesController {
private final ResourceManager resourceManager;
@GetMapping("/ui")
@SneakyThrows
@Authorize(merge = false)
public Flux<Object> getUIResources() {
return resourceManager
.getResources(UiResourceProvider.TYPE)
.map(resource->resource.as(HashMap.class));
}
@GetMapping("/{type}")
@SneakyThrows
public Flux<String> getResources(@PathVariable String type) {

View File

@@ -0,0 +1,2 @@
org.jetlinks.community.configuration.CommonConfiguration
org.jetlinks.community.configuration.UiResourceConfiguration

View File

@@ -1,14 +1,129 @@
message.device_message_handing=Message sent to device, processing...
message.value_cannot_be_empty=Value cannot be empty
message.value_max_valida_not_passed=The maximum value check fail
message.value_min_valida_not_passed=The minimum value check fail
message.value_range_valida_not_passed=The range value check fail
message.value_must_conform_regular_check=The value must conform to the check regular
message.validator_not_passed=Validator check fail
message.validator_is_passed=Validator check passed
message.term-type-and=and
message.term-type-or=or
error.duplicate_key_detail=Duplicate Data:{0}
error.data.referenced=The data has been used elsewhere
error.scene_rule_timer_cron_cannot_be_empty=The cron cannot be null
error.entity_template_exist=The entity template already exists
error.base_path_error=base-path error.\
format:{http/https}://{IP address of the server where the front-end is located}:{Front end exposed service port}/api
format:{http/https}://{IP address of the server where the front-end is located}:{Front end exposed service \
port}/{Front end agent api}
error.base_path_DNS_resolution_failed=base-path DNS resolution failed.\
format:{http/https}://{IP address of the server where the front-end is located}:{Front end exposed service port}/api
format:{http/https}://{IP address of the server where the front-end is located}:{Front end exposed service \
port}/{Front end agent api}
error.base_path_validate_request_timeout=base-path validate request timeout,\
Please check if the internal access of the server can be done normally using the base path address
error.base_path_host_error=The HOST for base path cannot be:{0},please use an IPV4 address or domain name.\
Example:192.168.x.x
error.jvm_error=System Internal error
error.user_binding_code_incorrect=The user binding code has expired or is incorrect
error.illegal_bind=illegal bind
error.jvm_error=System Internal error
error.system_dictionary_can_not_delete=Operation failed, system dictionary cannot be deleted
error.system_dictionary_can_not_update=Operation failed, system dictionary cannot be modify
message.timer_spec_desc_seperator=\u0020and\u0020
message.timer_spec_desc_everyday=everyday
message.timer_spec_desc_everyweek=on every {0}
message.timer_spec_desc_everymonth=on the {0} of each month
message.timer_spec_desc=Execute once {1} {0}
message.timer_spec_desc_period= every {1}{2} {0}
message.timer_spec_desc_period_duration=from {0} to {1}
message.timer_spec_desc_period_once=at {0}
message.timer_spec_desc_calendar=calendar trigger with {0} rule(s)
message.timer_spec_desc_week_1=monday
message.timer_spec_desc_week_2=tuesday
message.timer_spec_desc_week_3=wednesday
message.timer_spec_desc_week_4=thursday
message.timer_spec_desc_week_5=friday
message.timer_spec_desc_week_6=saturday
message.timer_spec_desc_week_7=Sunday
message.timer_spec_desc_month={0}th
message.timer_spec_desc_month_1=1st
message.timer_spec_desc_month_2=2nd
message.timer_spec_desc_month_3=3rd
message.timer_spec_desc_period_hours=\u0020hour
message.timer_spec_desc_period_minutes=\u0020minute
message.timer_spec_desc_period_seconds=\u0020second
error.the_template_is_enabled_and_cannot_be_deleted=The template[{0}] is enabled and cannot be deleted
error.exception.illegal_argument=Illegal argument
error.exception.unsupported_operation=Unsupported operation
error.exception.timeout=Timeout
error.internal_error=Internal error
message.system.constant.dict.name=classification of system constants
message.calendar.tags.dict.item.weekend=weekend
message.calendar.tags.dict.item.workday=workday
message.calendar.tags.dict.item.holiday=holiday
message.calendar.tags.dict.name=schedule tags
message.system.constant.dict.item.default.name=default
hswebframework.web.system.dictionary.schedule-tags=Calendar Tags
hswebframework.web.system.dictionary.item.workday=Workday
hswebframework.web.system.dictionary.item.weekend=Weekend
hswebframework.web.system.dictionary.item.holiday=Holiday
hswebframework.web.system.action.query=Query
hswebframework.web.system.action.save=Save
hswebframework.web.system.action.delete=Delete
hswebframework.web.system.action.upload-static=Static File
hswebframework.web.system.permission.system_environment=System Environment Parameters
hswebframework.web.system.permission.system-monitor=System Internal Monitoring
hswebframework.web.system.permission.system-resources=System Resources
hswebframework.web.system.permission.system_config=System Configuration Management
hswebframework.web.system.permission.entity-template=Resource Library(Old)
hswebframework.web.system.permission.system_constant=System Constant Configuration Management
hswebframework.web.system.permission.calendar-manager=Calendar Management
#FixedTermType
message.term_type_eq=Equals
message.term_type_neq=Not equal
message.term_type_notnull=Is not empty
message.term_type_isnull=Is empty
message.term_type_gt=Greater than
message.term_type_gte=Greater than or equal to
message.term_type_lt=Less than
message.term_type_lte=Less than or equal to
message.term_type_btw=Between
message.term_type_nbtw=Not between
message.term_type_in=In
message.term_type_nin=Not in
message.term_type_contains_all=Include all
message.term_type_contains_any=Include any
message.term_type_not_contains=Exclude
message.term_type_like=Like
message.term_type_nlike=Not like
message.term_type_time_gt_now=Time greater than now
message.term_type_time_lt_now=Time less than now
message.term_type_complex_exists=Complex exists
message.term_type_eq_desc={0} equals {1}
message.term_type_neq_desc={0} not equal to {1}
message.term_type_notnull_desc={0} is not null {1}
message.term_type_isnull_desc={0} is null {1}
message.term_type_gt_desc={0} is greater than {1}
message.term_type_gte_desc={0} is greater than or equal to {1}
message.term_type_lt_desc={0} is less than {1}
message.term_type_lte_desc={0} is less than or equal to {1}
message.term_type_btw_desc={0} is between {1}
message.term_type_nbtw_desc={0} is not between {1}
message.term_type_in_desc={0} is in {1}
message.term_type_nin_desc={0} is not in {1}
message.term_type_contains_all_desc={0} contains all in {1}
message.term_type_contains_any_desc={0} contains any in {1}
message.term_type_not_contains_desc={0} does not contain in {1}
message.term_type_like_desc={0} contains character {1}
message.term_type_nlike_desc={0} does not contain character {1}
message.term_type_time_gt_now_desc={0} is greater than {1} seconds from current time
message.term_type_time_lt_now_desc={0} is less than {1} seconds from current time
message.term_complex_exists_desc={0} is {1}
org.jetlinks.community.template.EntityTemplateState.enabled=Enabled
org.jetlinks.community.template.EntityTemplateState.disabled=Disabled

View File

@@ -1,8 +1,124 @@
message.device_message_handing=\u6D88\u606F\u5DF2\u53D1\u5F80\u8BBE\u5907\uFF0C\u5904\u7406\u4E2D...
error.data.referenced=\u6570\u636E\u5DF2\u7ECF\u88AB\u5176\u4ED6\u5730\u65B9\u4F7F\u7528
message.device_message_handing=\u6D88\u606F\u5DF2\u53D1\u5F80\u8BBE\u5907,\u5904\u7406\u4E2D...
message.value_cannot_be_empty=\u503C\u4E0D\u80FD\u4E3A\u7A7A
message.value_max_valida_not_passed=\u6700\u5927\u503C\u6821\u9A8C\u672A\u901A\u8FC7
message.value_min_valida_not_passed=\u6700\u5C0F\u503C\u6821\u9A8C\u672A\u901A\u8FC7
message.value_range_valida_not_passed=\u8303\u56F4\u503C\u6821\u9A8C\u672A\u901A\u8FC7
message.value_must_conform_regular_check=\u503C\u9700\u7B26\u5408\u6821\u9A8C\u6B63\u5219
message.validator_not_passed=\u68C0\u9A8C\u5668\u68C0\u67E5\u4E0D\u901A\u8FC7
message.validator_is_passed=\u68C0\u9A8C\u5668\u68C0\u67E5\u901A\u8FC7
message.term-type-and=\u5E76\u4E14
message.term-type-or=\u6216\u8005
error.duplicate_key_detail=\u91CD\u590D\u7684\u6570\u636E:{0}
error.base_path_error=base-path\u9519\u8BEF\uFF0C\u6B63\u786E\u683C\u5F0F:{http/https}://{\u524D\u7AEF\u6240\u5728\u670D\u52A1\u5668IP\u5730\u5740}:{\u524D\u7AEF\u66B4\u9732\u7684\u670D\u52A1\u7AEF\u53E3}/api
error.base_path_DNS_resolution_failed=base-path DNS\u89E3\u6790\u5931\u8D25\uFF0C\u6B63\u786E\u683C\u5F0F:{http/https}://{\u524D\u7AEF\u6240\u5728\u670D\u52A1\u5668IP\u5730\u5740}:{\u524D\u7AEF\u66B4\u9732\u7684\u670D\u52A1\u7AEF\u53E3}/api
error.data.referenced=\u6570\u636E\u5DF2\u7ECF\u88AB\u5176\u4ED6\u5730\u65B9\u4F7F\u7528
error.scene_rule_timer_cron_cannot_be_empty=cron\u8868\u8FBE\u5F0F\u4E0D\u80FD\u4E3A\u7A7A
error.entity_template_exist=\u5B9E\u4F53\u6A21\u677F\u5DF2\u5B58\u5728
error.base_path_error=base-path\u9519\u8BEF\uFF0C\u6B63\u786E\u683C\u5F0F:{http/https}://{\u524D\u7AEF\u6240\u5728\u670D\u52A1\u5668IP\u5730\u5740}:{\u524D\u7AEF\u66B4\u9732\u7684\u670D\u52A1\u7AEF\u53E3}/{\u524D\u7AEF\u4EE3\u7406api}
error.base_path_DNS_resolution_failed=base-path DNS\u89E3\u6790\u5931\u8D25\uFF0C\u6B63\u786E\u683C\u5F0F:{http/https}://{\u524D\u7AEF\u6240\u5728\u670D\u52A1\u5668IP\u5730\u5740}:{\u524D\u7AEF\u66B4\u9732\u7684\u670D\u52A1\u7AEF\u53E3}/{\u524D\u7AEF\u4EE3\u7406api}
error.base_path_validate_request_timeout=base-path\u8BF7\u6C42\u9A8C\u8BC1\u8D85\u65F6\uFF0C\u8BF7\u68C0\u67E5\u670D\u52A1\u5668\u5185\u90E8\u8BBF\u95EE\u662F\u5426\u80FD\u6B63\u5E38\u65B9\u5F0Fbase-path\u5730\u5740
error.base_path_host_error=base-path\u7684HOST\u4E0D\u80FD\u4E3A:{0}\uFF0C\u8BF7\u4F7F\u7528IPv4\u5730\u5740\u6216\u57DF\u540D.\u5982:192.168.x.x
error.jvm_error=\u7CFB\u7EDF\u5185\u90E8\u9519\u8BEF
error.user_binding_code_incorrect=\u7528\u6237\u7ED1\u5B9A\u7801\u5DF2\u8FC7\u671F\u6216\u9519\u8BEF
error.illegal_bind=\u975E\u6CD5\u7ED1\u5B9A
error.jvm_error=\u7CFB\u7EDF\u5185\u90E8\u9519\u8BEF
error.system_dictionary_can_not_delete=\u64CD\u4F5C\u5931\u8D25\uFF0C\u7CFB\u7EDF\u5B57\u5178\u65E0\u6CD5\u5220\u9664
error.system_dictionary_can_not_update=\u64CD\u4F5C\u5931\u8D25\uFF0C\u7CFB\u7EDF\u5B57\u5178\u7981\u6B62\u4FEE\u6539
message.timer_spec_desc_seperator=\uFF0C
message.timer_spec_desc_everyday=\u6BCF\u5929
message.timer_spec_desc_everyweek=\u6BCF\u5468{0}
message.timer_spec_desc_everymonth=\u6BCF\u6708{0}
message.timer_spec_desc={0} {1}\u6267\u884C1\u6B21
message.timer_spec_desc_period={0} \u6BCF{1}{2}
message.timer_spec_desc_period_duration={0}-{1}
message.timer_spec_desc_period_once={0}
message.timer_spec_desc_calendar=\u81EA\u5B9A\u4E49\u65E5\u5386 \u5171{0}\u4E2A\u89C4\u5219
message.timer_spec_desc_week_1=\u661F\u671F\u4E00
message.timer_spec_desc_week_2=\u661F\u671F\u4E8C
message.timer_spec_desc_week_3=\u661F\u671F\u4E09
message.timer_spec_desc_week_4=\u661F\u671F\u56DB
message.timer_spec_desc_week_5=\u661F\u671F\u4E94
message.timer_spec_desc_week_6=\u661F\u671F\u516D
message.timer_spec_desc_week_7=\u661F\u671F\u65E5
message.timer_spec_desc_month={0}\u53F7
message.timer_spec_desc_month_1={0}\u53F7
message.timer_spec_desc_month_2={0}\u53F7
message.timer_spec_desc_month_3={0}\u53F7
message.timer_spec_desc_period_hours=\u5C0F\u65F6
message.timer_spec_desc_period_minutes=\u5206\u949F
message.timer_spec_desc_period_seconds=\u79D2
error.the_template_is_enabled_and_cannot_be_deleted=\u6A21\u677F[{0}]\u5DF2\u542F\u7528\u65E0\u6CD5\u5220\u9664
error.exception.illegal_argument=\u53C2\u6570\u9519\u8BEF
error.exception.unsupported_operation=\u4E0D\u652F\u6301\u7684\u64CD\u4F5C
error.exception.timeout=\u8D85\u65F6
error.internal_error=\u7CFB\u7EDF\u5185\u90E8\u9519\u8BEF
message.system.constant.dict.name=\u7CFB\u7EDF\u5E38\u91CF\u5206\u7C7B
message.calendar.tags.dict.item.weekend=\u5468\u672B
message.calendar.tags.dict.item.workday=\u5DE5\u4F5C\u65E5
message.calendar.tags.dict.item.holiday=\u8282\u5047\u65E5
message.calendar.tags.dict.name=\u65E5\u5386\u6807\u7B7E
message.system.constant.dict.item.default.name=\u9ED8\u8BA4
hswebframework.web.system.dictionary.schedule-tags=\u65E5\u5386\u6807\u7B7E
hswebframework.web.system.dictionary.item.workday=\u5DE5\u4F5C\u65E5
hswebframework.web.system.dictionary.item.weekend=\u5468\u672B
hswebframework.web.system.dictionary.item.holiday=\u8282\u5047\u65E5
hswebframework.web.system.action.query=\u67E5\u8BE2
hswebframework.web.system.action.save=\u4FDD\u5B58
hswebframework.web.system.action.delete=\u5220\u9664
hswebframework.web.system.action.upload-static=\u9759\u6001\u6587\u4EF6
hswebframework.web.system.permission.system_environment=\u7CFB\u7EDF\u73AF\u5883\u53C2\u6570
hswebframework.web.system.permission.system-monitor=\u7CFB\u7EDF\u5185\u90E8\u76D1\u63A7
hswebframework.web.system.permission.system-resources=\u7CFB\u7EDF\u8D44\u6E90
hswebframework.web.system.permission.system_config=\u7CFB\u7EDF\u914D\u7F6E\u7BA1\u7406
hswebframework.web.system.permission.entity-template=\u8D44\u6E90\u5E93(\u65E7)
hswebframework.web.system.permission.system_constant=\u7CFB\u7EDF\u5E38\u91CF\u914D\u7F6E\u7BA1\u7406
hswebframework.web.system.permission.calendar-manager=\u65E5\u5386\u7BA1\u7406
#FixedTermType
message.term_type_eq=\u7B49\u4E8E
message.term_type_neq=\u4E0D\u7B49\u4E8E
message.term_type_notnull=\u4E0D\u4E3A\u7A7A
message.term_type_isnull=\u4E3A\u7A7A
message.term_type_gt=\u5927\u4E8E
message.term_type_gte=\u5927\u4E8E\u7B49\u4E8E
message.term_type_lt=\u5C0F\u4E8E
message.term_type_lte=\u5C0F\u4E8E\u7B49\u4E8E
message.term_type_btw=\u5728...\u4E4B\u95F4
message.term_type_nbtw=\u4E0D\u5728...\u4E4B\u95F4
message.term_type_in=\u5728...\u4E4B\u4E2D
message.term_type_nin=\u4E0D\u5728...\u4E4B\u4E2D
message.term_type_contains_all=\u5168\u90E8\u5305\u542B\u5728...\u4E4B\u4E2D
message.term_type_contains_any=\u4EFB\u610F\u5305\u542B\u5728...\u4E4B\u4E2D
message.term_type_not_contains=\u4E0D\u5305\u542B\u5728...\u4E4B\u4E2D
message.term_type_like=\u5305\u542B\u5B57\u7B26
message.term_type_nlike=\u4E0D\u5305\u542B\u5B57\u7B26
message.term_type_time_gt_now=\u8DDD\u79BB\u5F53\u524D\u65F6\u95F4\u5927\u4E8E...\u79D2
message.term_type_time_lt_now=\u8DDD\u79BB\u5F53\u524D\u65F6\u95F4\u5C0F\u4E8E...\u79D2
message.term_type_complex_exists=\u6EE1\u8DB3
message.term_type_eq_desc={0}\u7B49\u4E8E{1}
message.term_type_neq_desc={0}\u4E0D\u7B49\u4E8E{1}
message.term_type_notnull_desc={0}\u4E0D\u4E3A\u7A7A
message.term_type_isnull_desc={0}\u4E3A\u7A7A
message.term_type_gt_desc={0}\u5927\u4E8E{1}
message.term_type_gte_desc={0}\u5927\u4E8E\u7B49\u4E8E{1}
message.term_type_lt_desc={0}\u5C0F\u4E8E{1}
message.term_type_lte_desc={0}\u5C0F\u4E8E\u7B49\u4E8E{1}
message.term_type_btw_desc={0}\u5728{1}\u4E4B\u95F4
message.term_type_nbtw_desc={0}\u4E0D\u5728{1}\u4E4B\u95F4
message.term_type_in_desc={0}\u5728{1}\u4E4B\u4E2D
message.term_type_nin_desc={0}\u4E0D\u5728{1}\u4E4B\u4E2D
message.term_type_contains_all_desc={0}\u5168\u90E8\u5305\u542B\u5728{1}\u4E4B\u4E2D
message.term_type_contains_any_desc={0}\u4EFB\u610F\u5305\u542B\u5728{1}\u4E4B\u4E2D
message.term_type_not_contains_desc={0}\u4E0D\u5305\u542B\u5728{1}\u4E4B\u4E2D
message.term_type_like_desc={0}\u5305\u542B\u5B57\u7B26{1}
message.term_type_nlike_desc={0}\u4E0D\u5305\u542B\u5B57\u7B26{1}
message.term_type_time_gt_now_desc={0}\u8DDD\u79BB\u5F53\u524D\u65F6\u95F4\u5927\u4E8E{1}\u79D2
message.term_type_time_lt_now_desc={0}\u8DDD\u79BB\u5F53\u524D\u65F6\u95F4\u5C0F\u4E8E{1}\u79D2
message.term_complex_exists_desc={0}{1}\u6761\u4EF6
org.jetlinks.community.template.EntityTemplateState.enabled=\u6B63\u5E38
org.jetlinks.community.template.EntityTemplateState.disabled=\u7981\u7528

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>jetlinks-components</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

View File

@@ -1,11 +1,66 @@
package org.jetlinks.community.configure.cluster;
import org.apache.commons.collections4.MapUtils;
import org.springframework.core.env.Environment;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
public class Cluster {
static String ID = "default";
public static final String TAG_NAME = "name";
public static final String TAG_ID = "id";
private static String ID = "default";
private static String SAFE_ID = "default";
private static String NAME = "default";
private static Map<String, String> TAGS = Collections.emptyMap();
public static String id() {
return ID;
}
public static String name() {
return NAME;
}
public static String safeId() {
return SAFE_ID;
}
public static Map<String, String> tags() {
return TAGS;
}
public static boolean hasTag(Map<String, String> tag) {
if (MapUtils.isEmpty(tag)) {
return false;
}
for (Map.Entry<String, String> entry : tag.entrySet()) {
if (!Objects.equals(TAGS.get(entry.getKey()), entry.getValue())) {
return false;
}
}
return true;
}
public static synchronized void setup(Environment env){
Map<String, String> _tags = new HashMap<>(TAGS);
_tags.put("port",env.getProperty("server.port"));
}
public static synchronized void setup(String id, String name, Map<String, String> tags) {
ID = id;
SAFE_ID = ID.replaceAll("[\\s\\\\/:*?\"<>|]", "_");
NAME = name;
Map<String, String> _tags = new HashMap<>(tags);
_tags.putIfAbsent(TAG_NAME, name);
_tags.putIfAbsent(TAG_ID, id);
TAGS = Collections.unmodifiableMap(_tags);
}
}

View File

@@ -6,6 +6,7 @@ import org.jetlinks.core.trace.TraceHolder;
import org.springframework.boot.context.properties.ConfigurationProperties;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
@ConfigurationProperties(prefix = "jetlinks.cluster")
@@ -41,7 +42,7 @@ public class ClusterProperties {
public void setId(String id) {
this.id = id;
Cluster.ID = id;
Cluster.setup(id, id, Collections.emptyMap());
TraceHolder.setupGlobalName(id);
}

View File

@@ -1,17 +1,28 @@
package org.jetlinks.community.configure.device;
import lombok.Getter;
import lombok.Setter;
import io.netty.buffer.*;
import io.netty.util.ReferenceCountUtil;
import lombok.*;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.codec.binary.Base64;
import org.h2.mvstore.MVMap;
import org.h2.mvstore.MVStore;
import org.h2.mvstore.MVStoreException;
import org.h2.mvstore.WriteBuffer;
import org.h2.mvstore.type.BasicDataType;
import org.h2.mvstore.type.StringDataType;
import org.jetlinks.community.configure.cluster.Cluster;
import org.jetlinks.core.device.DeviceRegistry;
import org.jetlinks.core.device.DeviceState;
import org.jetlinks.core.device.session.DeviceSessionEvent;
import org.jetlinks.core.message.codec.DefaultTransport;
import org.jetlinks.core.rpc.RpcManager;
import org.jetlinks.core.server.session.DeviceSession;
import org.jetlinks.core.server.session.DeviceSessionProvider;
import org.jetlinks.core.server.session.LostDeviceSession;
import org.jetlinks.core.server.session.PersistentSession;
import org.jetlinks.core.utils.Reactors;
import org.jetlinks.core.utils.RecyclerUtils;
import org.jetlinks.community.codec.Serializers;
import org.jetlinks.supports.device.session.ClusterDeviceSessionManager;
import org.jetlinks.supports.utils.MVStoreUtils;
import org.springframework.beans.BeansException;
@@ -21,57 +32,141 @@ import org.springframework.context.ApplicationContextAware;
import org.springframework.data.util.Lazy;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.core.scheduler.Scheduler;
import reactor.core.scheduler.Schedulers;
import javax.annotation.Nonnull;
import java.io.File;
import javax.annotation.PreDestroy;
import java.io.*;
import java.lang.reflect.Array;
import java.nio.ByteBuffer;
import java.time.Duration;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
import java.util.function.Function;
import java.util.function.Supplier;
/**
* 基于<a href="https://h2database.com/html/mvstore.html">MvStore</a>的本地磁盘持久化设备会话管理器.
* <p>
* 实现{@link PersistentSession}接口的设备会话将持久化到磁盘,在平台重启后,将回复这些设备会话.
* 可通过实现{@link DeviceSessionProvider}来自定义设备会话的序列化.
*
* <p>
* 如果配置了{@link PersistenceDeviceSessionManager#recordHistory}=true,当设备上线时
* 将记录设备的历史状态,在平台重启后,将检查这些设备的状态,如果设备离线,将触发离线事件.
*
* <p>
* 优点: 非中心化,不依赖中心化的中间件,集群时,更利于横向拓展。在磁盘性能(SSD)充足的情况下,性能影响很小.
* <p>
* 缺点: 依赖磁盘IO,程序异常退出可能导致文件损坏而数据丢失.
*
* @author zhouhao
* @see PersistentSession
* @see DeviceSessionProvider
* @see org.jetlinks.core.server.session.DeviceSessionProviders
* @since 2.0
*/
@Slf4j
public class PersistenceDeviceSessionManager extends ClusterDeviceSessionManager implements CommandLineRunner, ApplicationContextAware {
private Supplier<DeviceRegistry> registry;
private MVMap<String, PersistentSessionEntity> repository;
protected Supplier<DeviceRegistry> registry;
private MVMap<String, PersistentSessionData> repository;
private MVMap<String, Long> history;
private final Scheduler scheduler = Schedulers.newSingle("device-session-persistence");
@Getter
@Setter
private String filePath;
@Getter
@Setter
//刷新间隔,将session进行持久化的间隔周期.
private Duration flushInterval = Duration.ofMinutes(10);
//记录历史上线设备,重启后将对这些设备进行状态检查,如果离线将触发离线事件.
@Getter
@Setter
private boolean recordHistory = true;
//启动后延迟同步设备状态,默认10秒
@Getter
@Setter
private Duration stateSyncDelay = Duration.ofSeconds(10);
public PersistenceDeviceSessionManager(RpcManager rpcManager) {
super(rpcManager);
}
static MVMap<String, PersistentSessionEntity> initStore(String file) {
MVStore store =
MVStoreUtils.open(
void initStore(String file) {
MVStoreUtils
.open(
new File(file),
"device-session",
builder -> {
return builder.cacheSize(1);
builder -> builder.cacheSize(32),
store -> {
repository =
store.openMap("device-session-2", new MVMap
.Builder<String, PersistentSessionData>()
.keyType(StringDataType.INSTANCE)
.valueType(new PersistentSessionDataType()));
history = MVStoreUtils.openMap(store, "history-2", new MVMap.Builder<>());
//兼容旧数据
if (store.getMapNames().contains("device-session")) {
restore(
MVStoreUtils.openMap(store, "device-session", new MVMap.Builder<>())
);
store.removeMap("device-session");
}
return null;
});
return MVStoreUtils.openMap(store, "device-session", new MVMap.Builder<>());
}
private synchronized void initRepository() {
try {
if (repository != null && !repository.store.isClosed()) {
repository
.store
.close(1000);
}
} catch (Throwable ignore) {
}
initStore(filePath);
}
private void restore(MVMap<String, PersistentSessionEntity> repo) {
for (Map.Entry<String, PersistentSessionEntity> e : repo.entrySet()) {
byte[] data = Base64.decodeBase64(e.getValue().getSessionBase64());
repository
.put(e.getKey(), new PersistentSessionData(
e.getValue().getProvider(),
e.getValue().getDeviceId(),
data));
repo.remove(e.getKey());
}
}
@Override
public void init() {
super.init();
if (filePath == null) {
filePath = "./data/sessions-" + (Cluster
.id()
.replace(":", "_")
.replace("/", ""));
filePath = "./data/sessions/" + (Cluster.safeId());
}
repository = initStore(filePath);
initRepository();
if (!flushInterval.isZero() && !flushInterval.isNegative()) {
disposable.add(
Flux
.interval(flushInterval)
.interval(flushInterval, Schedulers.boundedElastic())
.onBackpressureDrop()
.concatMap(ignore -> Flux
.fromIterable(localSessions.values())
@@ -87,29 +182,86 @@ public class PersistenceDeviceSessionManager extends ClusterDeviceSessionManager
}
disposable.add(
listenEvent(event -> {
//移除持久化的会话
if (event.getType() == DeviceSessionEvent.Type.unregister
&& event.getSession().isWrapFrom(PersistentSession.class)) {
return removePersistentSession(
event.getSession().unwrap(PersistentSession.class)
);
}
return Mono.empty();
})
listenEvent(event -> Mono
.deferContextual(ctx -> {
if (ctx.hasKey(PersistenceDeviceSessionManager.class)) {
return Mono.empty();
}
//持久化
if (event.getSession().isWrapFrom(PersistentSession.class)) {
if (event.getType() == DeviceSessionEvent.Type.unregister) {
return removePersistentSession(event.getSession().unwrap(PersistentSession.class));
}
} else if (recordHistory) {
//记录历史设备
if (event.getType() == DeviceSessionEvent.Type.register) {
return this
.operateInStore(
() -> history,
history -> history.put(event.getSession().getDeviceId(),
event.getSession().connectTime())
)
.then();
}
if (event.getType() == DeviceSessionEvent.Type.unregister && !isShutdown()) {
return this
.operateInStore(
() -> history,
history -> history.remove(event.getSession().getDeviceId())
)
.then();
}
}
return Mono.empty();
}))
);
//disposable.add(scheduler);
}
private <K, V> Mono<V> operateInStore(Supplier<MVMap<K, V>> map, Function<MVMap<K, V>, V> function) {
return Mono
.fromCallable(() -> {
int retry = 0;
do {
try {
return function.apply(map.get());
} catch (MVStoreException e) {
initRepository();
}
} while (retry++ == 0);
return function.apply(map.get());
})
.subscribeOn(scheduler);
}
@Override
@PreDestroy
public void shutdown() {
super.shutdown();
Flux.fromIterable(localSessions.values())
.filter(ref -> ref.loaded != null)
.filter(ref -> ref.loaded.isWrapFrom(PersistentSession.class))
log.info("Persistent and close session");
Map<String, DeviceSessionRef> sessions = new HashMap<>(localSessions);
localSessions.clear();
Flux.fromIterable(sessions.values())
.filter(ref -> ref.loaded != null && ref.loaded.isWrapFrom(PersistentSession.class))
.map(ref -> ref.loaded.unwrap(PersistentSession.class))
.as(this::tryPersistent)
.then(
Flux.fromIterable(sessions.values())
.filter(ref -> ref.loaded != null && !ref.loaded.isWrapFrom(PersistentSession.class))
.flatMap(ref -> closeSessionSafe(ref.loaded))
.then()
)
.block();
repository.store.close(-1);
log.info("Compact and close local session store {}", filePath);
if (repository.size() < 10_0000) {
repository.store.close(-1);
} else {
repository.store.close(10_000);
}
}
@Override
@@ -129,28 +281,48 @@ public class PersistenceDeviceSessionManager extends ClusterDeviceSessionManager
Mono<Void> tryPersistent(Flux<PersistentSession> sessions) {
return sessions
.flatMap(session -> PersistentSessionEntity.from(getCurrentServerId(), session, registry.get()))
.distinct(PersistentSessionEntity::getId)
.doOnNext(e -> {
log.debug("persistent device[{}] session", e.getDeviceId());
repository.put(e.getDeviceId(), e);
})
.flatMap(session -> PersistentSessionData.of(registry.get(), session))
.concatMap(e -> this
.operateInStore(
() -> repository,
repository -> {
log.debug("Persistent device[{}] session", e.getDeviceId());
repository.put(e.getDeviceId(), e);
return null;
}))
.onErrorResume(err -> {
log.warn("persistent session error", err);
log.warn("Persistent session error", err);
return Mono.empty();
})
.then();
}
Mono<Void> resumeSession(PersistentSessionEntity entity) {
Mono<Void> resumeSession(PersistentSessionData entity) {
return entity
.toSession(registry.get())
.filterWhen(session -> {
if (session.getOperator() == null) {
return Reactors.ALWAYS_FALSE;
}
return session
.getOperator()
.getState()
.map(state -> {
//设备已经离线?
if (!Objects.equals(state, DeviceState.online)) {
repository.remove(session.getDeviceId());
return false;
}
return true;
});
})
.doOnNext(session -> {
log.debug("resume session[{}]", session.getDeviceId());
localSessions.putIfAbsent(session.getDeviceId(),
new DeviceSessionRef(session.getDeviceId(),
this,
session));
localSessions.putIfAbsent(
session.getDeviceId(),
new DeviceSessionRef(session.getDeviceId(),
this,
session));
})
.onErrorResume((err) -> {
log.debug("resume session[{}] error", entity.getDeviceId(), err);
@@ -160,20 +332,196 @@ public class PersistenceDeviceSessionManager extends ClusterDeviceSessionManager
}
Mono<Void> removePersistentSession(PersistentSession session) {
repository.remove(session.getId());
return Mono.empty();
return this
.operateInStore(
() -> repository,
history -> repository.remove(session.getDeviceId())
)
.then();
}
@Override
public void run(String... args) throws Exception {
Flux.fromIterable(repository.values())
.flatMap(this::resumeSession)
.subscribe();
scheduler.schedule(() -> {
int retry = 0;
long time = System.currentTimeMillis();
do {
try {
Flux.fromIterable(repository.values())
.flatMap(this::resumeSession)
//延迟同步设备状态
.then(Mono.delay(stateSyncDelay))
.then(
//尝试同步历史设备状态
Flux.fromIterable(history.entrySet())
.flatMap(
idAndTime -> registry
.get()
.getDevice(idAndTime.getKey())
.flatMap(device -> device
.checkState()
.flatMap(state -> {
//曾经在线的设备离线了.
if (Objects.equals(DeviceState.offline, state)) {
log.debug("device [{}] session lost", idAndTime.getKey());
return fireEvent(
DeviceSessionEvent.of(
DeviceSessionEvent.Type.unregister,
new LostDeviceSession(idAndTime.getKey(),
device,
DefaultTransport.TCP,
idAndTime.getValue()),
false
)
);
}
return Mono.empty();
})
)
.then(Mono.fromCallable(() -> history.remove(idAndTime.getKey()))
.subscribeOn(Schedulers.boundedElastic()))
,
8)
.contextWrite(ctx -> ctx.put(PersistenceDeviceSessionManager.class, this))
.then()
)
.doFinally(ignore -> log.info(
"load device session cost {}ms",
System.currentTimeMillis() - time - stateSyncDelay.toMillis()))
.subscribe();
break;
} catch (MVStoreException e) {
initRepository();
}
} while (retry++ == 0);
});
}
@Override
public void setApplicationContext(@Nonnull ApplicationContext applicationContext) throws BeansException {
this.registry = Lazy.of(() -> applicationContext.getBean(DeviceRegistry.class));
}
@SneakyThrows
static ObjectInput createInput(ByteBuf buffer) {
return Serializers.getDefault().createInput(new ByteBufInputStream(buffer, true));
}
@SneakyThrows
static ObjectOutput createOutput(ByteBuf buffer) {
return Serializers.getDefault().createOutput(new ByteBufOutputStream(buffer));
}
@SuppressWarnings("all")
static class PersistentSessionDataType extends BasicDataType<PersistentSessionData> {
@Override
public int getMemory(PersistentSessionData obj) {
return obj.data.length;
}
@Override
@SneakyThrows
public void write(WriteBuffer buff, PersistentSessionData data) {
ByteBuf buffer = ByteBufAllocator.DEFAULT.buffer();
try (ObjectOutput output = createOutput(buffer)) {
data.writeExternal(output);
output.flush();
buff.put(buffer.nioBuffer());
} finally {
ReferenceCountUtil.safeRelease(buffer);
}
}
@Override
@SneakyThrows
public void write(WriteBuffer buff, Object obj, int len) {
ByteBuf buffer = ByteBufAllocator.DEFAULT.buffer();
try (ObjectOutput output = createOutput(buffer)) {
for (int i = 0; i < len; i++) {
@SuppressWarnings("all")
PersistentSessionData buf = ((PersistentSessionData) Array.get(obj, i));
buf.writeExternal(output);
}
output.flush();
buff.put(buffer.nioBuffer());
} finally {
ReferenceCountUtil.safeRelease(buffer);
}
}
@Override
@SneakyThrows
public void read(ByteBuffer buff, Object obj, int len) {
try (ObjectInput input = createInput(Unpooled.wrappedBuffer(buff))) {
for (int i = 0; i < len; i++) {
PersistentSessionData data = new PersistentSessionData();
data.readExternal(input);
Array.set(obj, i, data);
}
}
}
@Override
@SneakyThrows
public PersistentSessionData read(ByteBuffer buff) {
PersistentSessionData data = new PersistentSessionData();
try (ObjectInput input = createInput(Unpooled.wrappedBuffer(buff))) {
data.readExternal(input);
}
return data;
}
@Override
public PersistentSessionData[] createStorage(int size) {
return new PersistentSessionData[size];
}
}
@AllArgsConstructor
@NoArgsConstructor
@Getter
private static class PersistentSessionData implements Externalizable {
private String provider;
private String deviceId;
private byte[] data;
public Mono<PersistentSession> toSession(DeviceRegistry registry) {
DeviceSessionProvider provider = DeviceSessionProvider
.lookup(this.provider)
.orElseGet(UnknownDeviceSessionProvider::getInstance);
return provider.deserialize(data, registry);
}
public static Mono<PersistentSessionData> of(DeviceRegistry registry, PersistentSession session) {
DeviceSessionProvider provider = DeviceSessionProvider
.lookup(session.getProvider())
.orElseGet(UnknownDeviceSessionProvider::getInstance);
return provider
.serialize(session, registry)
.map(data -> {
PersistentSessionData persistentSessionData = new PersistentSessionData();
persistentSessionData.provider = session.getProvider();
persistentSessionData.deviceId = session.getDeviceId();
persistentSessionData.data = data;
return persistentSessionData;
});
}
@Override
public void writeExternal(ObjectOutput out) throws IOException {
out.writeUTF(provider);
out.writeInt(data.length);
out.write(data);
}
@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
provider = RecyclerUtils.intern(in.readUTF());
data = new byte[in.readInt()];
in.readFully(data);
}
}
}

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>jetlinks-components</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

View File

@@ -0,0 +1 @@
hswebframework.web.system.permission.dashboard=Dashboard

View File

@@ -0,0 +1 @@
hswebframework.web.system.permission.dashboard=\u4EEA\u8868\u76D8

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>jetlinks-components</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

View File

@@ -6,6 +6,7 @@ import org.jetlinks.community.elastic.search.index.ElasticSearchIndexManager;
import org.jetlinks.community.things.data.operations.ColumnModeDDLOperationsBase;
import org.jetlinks.community.things.data.operations.DataSettings;
import org.jetlinks.community.things.data.operations.MetricBuilder;
import org.jetlinks.core.things.ThingMetadata;
import reactor.core.publisher.Mono;
import java.util.List;

View File

@@ -35,4 +35,9 @@ class ElasticSearchRowModeDDLOperations extends RowModeDDLOperationsBase {
return indexManager
.putIndex(new DefaultElasticSearchIndexMetadata(metric, properties));
}
@Override
protected boolean isOnlySupportsOneObjectOrArrayProperty() {
return true;
}
}

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>jetlinks-components</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

View File

@@ -13,18 +13,22 @@ import org.jetlinks.core.server.session.ChildrenDeviceSession;
import org.jetlinks.core.server.session.DeviceSession;
import org.jetlinks.core.server.session.KeepOnlineSession;
import org.jetlinks.core.server.session.LostDeviceSession;
import org.jetlinks.core.utils.Reactors;
import org.jetlinks.community.PropertyConstants;
import org.jetlinks.supports.server.DecodedClientMessageHandler;
import org.springframework.util.StringUtils;
import reactor.core.publisher.Mono;
import reactor.util.context.Context;
import javax.validation.constraints.NotNull;
import java.time.Duration;
import java.util.Objects;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.function.Supplier;
import static org.jetlinks.core.message.Headers.ignoreIfOffline;
/**
* 设备网关消息处理,会话管理工具类,用于统一封装对设备消息和会话的处理逻辑
*
@@ -83,7 +87,7 @@ public class DeviceGatewayHelper {
return handleDeviceMessage(message, sessionBuilder, sessionConsumer, () -> Mono.fromRunnable(deviceNotFoundCallback));
}
private Mono<Void> handleChildrenDeviceMessage(String deviceId, DeviceMessage children) {
private void handleChildrenDeviceMessage(String deviceId, DeviceMessage children, HandlerContext ctx) {
//设备状态检查,断开设备连接的消息都忽略
//这些消息属于状态管理,通常是用来自定义子设备状态的,所以这些消息都忽略处理会话
if (deviceId == null
@@ -92,13 +96,13 @@ public class DeviceGatewayHelper {
|| children instanceof DisconnectDeviceMessage
|| children instanceof DisconnectDeviceMessageReply
|| children.getHeaderOrDefault(Headers.ignoreSession)) {
return Mono.empty();
return;
}
//子设备回复失败的也忽略
if (children instanceof DeviceMessageReply) {
DeviceMessageReply reply = ((DeviceMessageReply) children);
if (!reply.isSuccess()) {
return Mono.empty();
return;
}
}
String childrenId = children.getDeviceId();
@@ -107,27 +111,44 @@ public class DeviceGatewayHelper {
if (children instanceof DeviceOfflineMessage || children instanceof DeviceUnRegisterMessage) {
//注销会话,这里子设备可能会收到多次离线消息
//注销会话一次离线,消息网关转发子设备消息一次
return sessionManager
.remove(childrenId, removeSessionOnlyLocal(children))
.doOnNext(total -> {
if (total > 0 && children instanceof DeviceOfflineMessage) {
children.addHeader(Headers.ignore, true);
}
})
.then();
//先执行移除子设备会话,防止header设置失败
ctx.before(
sessionManager
.remove(childrenId, removeSessionOnlyLocal(children))
.doOnNext(total -> {
//移除了会话会触发离线消息,忽略掉本次的消息.
if (total > 0 && children instanceof DeviceOfflineMessage) {
children.addHeaderIfAbsent(Headers.ignore, true);
}
//没有会话被移除(已经离线),但是手动指定了忽略离线消息.
if (total == 0 && children.getHeaderOrDefault(ignoreIfOffline)) {
children.addHeader(Headers.ignore, true);
}
})
.then()
);
} else {
//子设备上线
if (children instanceof DeviceOnlineMessage) {
children.addHeader(Headers.ignore, true);
//没有标记force,则忽略上线消息,避免产生重复的上线消息.
if (!children.getHeader(Headers.force).orElse(false)) {
children.addHeaderIfAbsent(Headers.ignore, true);
}
}
//子设备会话处理
Mono<DeviceSession> sessionHandler = sessionManager
Mono<DeviceSession> sessionHandler = children.getHeaderOrDefault(Headers.ignoreSession)
? Mono.empty()
: sessionManager
.getSession(deviceId)
.flatMap(parentSession -> this
.createOrUpdateSession(childrenId,
children,
child -> Mono.just(new ChildrenDeviceSession(childrenId, parentSession, child)),
Mono::empty)
.createOrUpdateSession(
childrenId,
children,
child -> {
//新创建了的会话?
return Mono.just(new ChildrenDeviceSession(childrenId, parentSession, child));
},
Mono::empty)
.doOnNext(session -> {
if (session.isWrapFrom(ChildrenDeviceSession.class)) {
ChildrenDeviceSession childrenSession = session.unwrap(ChildrenDeviceSession.class);
@@ -141,17 +162,21 @@ public class DeviceGatewayHelper {
//子设备注册
if (isDoRegister(children)) {
return this
.getDeviceForRegister(children.getDeviceId())
.flatMap(device -> device
//没有配置状态自管理才自动上线
.getSelfConfig(DeviceConfigKey.selfManageState)
.defaultIfEmpty(false)
.filter(Boolean.FALSE::equals))
.flatMap(ignore -> sessionHandler)
.then();
ctx.after(
this
.getDeviceForRegister(children.getDeviceId())
.flatMap(device -> device
//没有配置状态自管理才自动上线
.getSelfConfig(DeviceConfigKey.selfManageState)
.defaultIfEmpty(false)
.filter(Boolean.FALSE::equals))
.flatMap(ignore -> sessionHandler)
.then()
);
} else {
ctx.after(sessionHandler.then());
}
return sessionHandler.then();
}
}
@@ -163,26 +188,26 @@ public class DeviceGatewayHelper {
if (!StringUtils.hasText(deviceId)) {
return Mono.empty();
}
Mono<DeviceOperator> then = null;
HandlerContext ctx = new HandlerContext();
boolean doHandle = true;
Context context = Context.of(DeviceMessage.class, message);
//子设备消息
if (message instanceof ChildDeviceMessage) {
DeviceMessage childrenMessage = (DeviceMessage) ((ChildDeviceMessage) message).getChildDeviceMessage();
then = handleChildrenDeviceMessage(deviceId, childrenMessage)
.then(registry.getDevice(deviceId));
handleChildrenDeviceMessage(deviceId, childrenMessage, ctx);
}
//子设备消息回复
else if (message instanceof ChildDeviceMessageReply) {
DeviceMessage childrenMessage = (DeviceMessage) ((ChildDeviceMessageReply) message).getChildDeviceMessage();
then = handleChildrenDeviceMessage(deviceId, childrenMessage)
.then(registry.getDevice(deviceId));
handleChildrenDeviceMessage(deviceId, childrenMessage, ctx);
}
//设备离线消息
else if (message instanceof DeviceOfflineMessage) {
return sessionManager
.remove(deviceId, removeSessionOnlyLocal(message))
.flatMap(l -> {
if (l == 0) {
if (l == 0 && !message.getHeaderOrDefault(ignoreIfOffline)) {
return registry
.getDevice(deviceId)
.flatMap(device -> handleMessage(device, message));
@@ -190,7 +215,7 @@ public class DeviceGatewayHelper {
return Mono.empty();
})
.then(registry.getDevice(deviceId))
.contextWrite(Context.of(DeviceMessage.class, message));
.contextWrite(context);
}
//设备上线消息,不发送到messageHandler,防止设备上线存在重复消息
else if (message instanceof DeviceOnlineMessage) {
@@ -199,30 +224,37 @@ public class DeviceGatewayHelper {
.orElse(false);
}
if (then == null) {
then = registry.getDevice(deviceId);
}
//忽略会话管理,比如一个设备存在多种接入方式时,其中一种接入方式收到的消息设置忽略会话来防止会话冲突
if (message.getHeaderOrDefault(Headers.ignoreSession)) {
if (!isDoRegister(message)) {
return handleMessage(null, message)
.then(then);
}
return then;
// if (!isDoRegister(message)) {
return ctx
.execute(handleMessage(null, message))
.then(registry.getDevice(deviceId))
.contextWrite(context);
// }
// return ctx
// .execute(Mono.empty())
// .then(registry.getDevice(deviceId))
// .contextWrite(context);
}
if (doHandle) {
then = handleMessage(null, message)
.then(then);
ctx.after(handleMessage(null, message));
}
return this
.createOrUpdateSession(deviceId, message, sessionBuilder, deviceNotFoundCallback)
.flatMap(sessionConsumer)
.then(then)
.contextWrite(Context.of(DeviceMessage.class, message));
return ctx
.execute(
this
.createOrUpdateSession(deviceId, message, sessionBuilder, deviceNotFoundCallback)
.flatMap(sessionConsumer)
)
.then(registry.getDevice(deviceId))
.contextWrite(context);
// return this
// .createOrUpdateSession(deviceId, message, sessionBuilder, deviceNotFoundCallback)
// .flatMap(sessionConsumer)
// .then(then)
// .contextWrite(context);
}
@@ -272,7 +304,8 @@ public class DeviceGatewayHelper {
return Mono.empty();
}),
session -> updateSession(session, message, sessionBuilder))))
.flatMap(Function.identity());
.flatMap(Function.identity())
.map(session -> handleSession(message, session));
}
private Mono<DeviceOperator> getDeviceForRegister(String deviceId) {
@@ -343,6 +376,17 @@ public class DeviceGatewayHelper {
: after;
}
private DeviceSession handleSession(DeviceMessage message, DeviceSession session) {
//尝试设置ignoreParent
if (session.isWrapFrom(KeepOnlineSession.class)) {
message
.getHeader(Headers.keepOnlineIgnoreParent)
.ifPresent(session.unwrap(KeepOnlineSession.class)::setIgnoreParent);
}
return session;
}
private static void applySessionKeepaliveTimeout(DeviceMessage msg, DeviceSession session) {
Integer timeout = msg.getHeaderOrElse(Headers.keepOnlineTimeoutSeconds, () -> null);
if (null != timeout) {
@@ -412,5 +456,57 @@ public class DeviceGatewayHelper {
}
/**
* 校验设备消息的网关ID
*
* @param accessId 当前网关ID
* @param message 设备消息
* @return 是否一致
*/
public Mono<Boolean> checkAccessId(@NotNull String accessId, DeviceMessage message) {
if (message.getHeaderOrDefault(Headers.multiGateway)) {
return Reactors.ALWAYS_TRUE;
}
return registry
.getDevice(message.getDeviceId())
.flatMap(operator -> operator.getConfig(PropertyConstants.accessId))
.map(accessId::equals)
.defaultIfEmpty(true);
}
private static class HandlerContext {
Mono<Void> before;
Mono<Void> after;
public void before(Mono<Void> before) {
if (this.before == null) {
this.before = before;
} else {
this.before = before.then(this.before);
}
}
public void after(Mono<Void> after) {
if (this.after == null) {
this.after = after;
} else {
this.after = this.after.then(after);
}
}
public Mono<Void> execute(Mono<Void> executor) {
Mono<Void> task = executor;
if (before != null) {
task = before.then(task);
}
if (after != null) {
task = task.then(after);
}
return task;
}
}
}

View File

@@ -1,22 +1,19 @@
package org.jetlinks.community.gateway.supports;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Generated;
import lombok.Getter;
import lombok.Setter;
import org.jetlinks.core.ProtocolSupport;
import org.jetlinks.community.ValueObject;
import javax.validation.constraints.NotBlank;
import java.util.HashMap;
import java.util.Map;
/**
* 设备网关属性外观类
* <p>
* 转换设备网关属性数据
* </p>
*
* @author zhouhao
*/
@Getter
@Setter
@Generated
public class DeviceGatewayProperties implements ValueObject {
private String id;
@@ -25,16 +22,40 @@ public class DeviceGatewayProperties implements ValueObject {
private String description;
/**
* 设备接入网关提供商标识
*
* @see DeviceGatewayProvider#getId()
*/
@NotBlank
private String provider;
@Schema(description = "接入通道ID,如网络组件ID")
@NotBlank
private String channelId;
/**
* @see ProtocolSupport#getId()
*/
@Schema(description = "接入使用的消息协议")
private String protocol;
/**
* 通信协议
*
* @see org.jetlinks.core.message.codec.DefaultTransport
*/
private String transport;
/**
* 网关配置信息,由{@link this#provider}决定
*/
private Map<String, Object> configuration = new HashMap<>();
private Map<String,Object> configuration=new HashMap<>();
/**
* 是否启用
*/
private boolean enabled = true;
@Override
public Map<String, Object> values() {

View File

@@ -20,4 +20,8 @@ public interface DeviceGatewayPropertiesManager {
Flux<DeviceGatewayProperties> getPropertiesByChannel(String channel);
default Flux<DeviceGatewayProperties> getAllProperties() {
return Flux.empty();
}
}

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>jetlinks-components</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
@@ -18,7 +18,7 @@
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.11.0</version>
<version>2.15.1</version>
</dependency>
<dependency>
@@ -80,4 +80,4 @@
</dependency>
</dependencies>
</project>
</project>

View File

@@ -58,7 +58,7 @@ public class DefaultImportExportService implements ImportExportService {
.zip(fileManager
.read(fileId)
.as(DataBufferUtils::join)
.map(DataBuffer::asInputStream),
.map(buffer->buffer.asInputStream(true)),
fileManager.getFile(fileId))
.flatMapMany(t2 -> read(t2.getT1(), t2.getT2().getExtension(), wrapper));
}

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>jetlinks-components</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>network-component</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

View File

@@ -73,8 +73,11 @@ class HttpDeviceSession implements DeviceSession {
@Override
public Mono<Boolean> send(EncodedMessage encodedMessage) {
//未建立websocket链接,不支持此类消息.
if(websocket == null){
return Mono.error(new DeviceOperationException(ErrorCode.UNSUPPORTED_MESSAGE));
if (websocket == null) {
return Mono.error(new DeviceOperationException.NoStackTrace(ErrorCode.UNSUPPORTED_MESSAGE));
}
if (!websocket.isAlive()) {
return Mono.error(new DeviceOperationException.NoStackTrace(ErrorCode.CONNECTION_LOST));
}
if (encodedMessage instanceof WebSocketMessage) {
return websocket
@@ -99,7 +102,10 @@ class HttpDeviceSession implements DeviceSession {
@Override
public void close() {
//断开websocket连接
if (websocket != null) {
websocket.close().subscribe();
}
}
@Override

View File

@@ -5,7 +5,7 @@
<parent>
<artifactId>network-component</artifactId>
<groupId>org.jetlinks.community</groupId>
<version>2.2.0-SNAPSHOT</version>
<version>2.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

Some files were not shown because too many files have changed in this diff Show More