feat: DingTalk media channel, OAuth2 SSO, multi-tenant management & Identity architecture adaptation#258
Open
nap-liu wants to merge 84 commits intodataelement:mainfrom
Open
Conversation
added 30 commits
March 29, 2026 02:56
…& UI polish Backend: - DingTalk: full media send/receive (image/file/voice/video) - DingTalk: thinking reaction indicator - DingTalk: auto-reconnect with exponential backoff - DingTalk: user source tracking + sender_nick display - /new command for DingTalk and Feishu session reset - secrets.md: creator-only file with redaction pipeline - Sensitive data sanitization (WebSocket, activity logs, A2A history) - SQL execute tool (MySQL/PostgreSQL/SQLite) - Multi-turn image context re-hydration - Agent seeding only on new company creation - Fix upstream Tenant.custom_domain -> sso_domain references Frontend: - Replace all emoji with Lucide React icons - Add secrets section to Mind tab - Chat tab moved to first position as default - i18n cleanup (zh + en)
…dia-support # Conflicts: # frontend/src/pages/EnterpriseSettings.tsx
…dia-support # Conflicts: # backend/app/services/registration_service.py # backend/app/services/sso_service.py
…dia-support # Conflicts: # backend/app/api/feishu.py
- Add DingTalkTokenManager singleton with per-app_key caching - Token cached for 7200s, auto-refresh 300s before expiry - Concurrency-safe with asyncio.Lock (prevents duplicate refresh) - Replace 4 independent token functions across dingtalk_stream.py, dingtalk_reaction.py, and dingtalk_service.py - Reduces token API calls from 5-6 per message to max 1
…_llm call _call_agent_llm (defined in feishu.py) does not accept context_size. The history is already truncated by SQL .limit(ctx_size) before the call, so passing context_size was both unnecessary and caused TypeError.
…截断 - websocket 查询用 ctx_size 替代硬编码 .limit(20) - 各通道 fallback 默认值统一为 100
- Tenant model: new subdomain_prefix field (String(50), unique, indexed) - Migration: add_subdomain_prefix (merges add_tool_source + merge_upstream_and_local heads) - domain.py: fallback chain Level 2 - subdomain_prefix + global hostname - tenants.py: check-prefix API, resolve-by-domain subdomain matching, schema updates (TenantOut/TenantUpdate), update_tenant validation - AdminCompanies: EditCompanyModal with subdomain prefix input + availability check - EnterpriseSettings: OrgTab Company URL display (read-only)
Prevent duplicate user creation when the same person uses both DingTalk bot and SSO login. Match chain: username -> unionId (org_members) -> mobile -> email -> create new. - Add _get_corp_access_token() with in-memory cache (2h, refresh 5min early) - Add _get_dingtalk_user_detail() for corp API user lookup - Load ChannelConfig early for API calls before user matching - Graceful degradation: API failure falls back to creating new user
# Conflicts: # backend/app/api/dingtalk.py
…nput from EditCompanyModal - CompanyStats model now includes subdomain_prefix field - list_companies endpoint now returns subdomain_prefix from tenant - EditCompanyModal: removed the sso_domain (Custom Access Domain) input as subdomain_prefix + global domain covers this use case - handleSave no longer sends sso_domain in payload
The backend _sync_tenant_sso_state() already auto-manages sso_enabled based on active identity providers. The manual toggle was redundant and caused confusion (users could configure SSO providers but forget to enable the toggle). Changes: - AdminCompanies.tsx: remove ssoEnabled state/checkbox from EditCompanyModal, remove sso_enabled from updateCompany API call, update description text - EnterpriseSettings.tsx: remove SSO on/off toggle from OrgTab SsoStatus, keep custom domain field always visible, remove sso_enabled from save payload
- backend: auto-generate subdomain_prefix from slug on company creation - backend: add _generate_subdomain_prefix helper (strips random hex suffix) - backend: add /tenants/check-slug API endpoint - backend: TenantUpdate now accepts slug field - backend: update_tenant validates + updates slug (format, reserved, uniqueness) - frontend: EditCompanyModal refactored with slug edit + domain config section - frontend: slug input with blur check-slug availability - frontend: domain config section with clearer headings, no emoji icons
… preview URL fallback - StatusBadge now uses t() for checking/available/taken text - Subdomain prefix input placeholder now shows current company slug instead of hardcoded "acme" - Preview URL falls back to slug when subdomain_prefix is empty - Added admin.* i18n keys to en.json and zh.json (slug, slugDesc, domainConfig, domainConfigDesc, subdomainPrefix, prefixAvailable, prefixTaken, prefixChecking, plus backfill of other admin keys missing from en.json)
…pport merge) The _get_corp_access_token() and _get_dingtalk_user_detail() helpers and the 6-step user matching chain were accidentally dropped when feature/ dingtalk-media-support was merged via 2cb741a. This restores them so that DingTalk bot messages match SSO users (via unionId/mobile/email) instead of always creating dingtalk_xxx dummy accounts.
…tegy
- Fix _get_corp_access_token: use GET instead of POST (errcode 43001)
- Add sender_id parameter to process_dingtalk_message
- Redesign 5-step user matching: local org_members first, API last
Step 1: sender_id -> org_members.external_id (fastest, no API)
Step 2: sender_staff_id -> org_members.external_id (compat)
Step 3: username = dingtalk_{staffId} (compat with old users)
Step 4: DingTalk API -> unionId/mobile/email matching (first time only)
Step 5: Create new user
- Auto-create org_member record after matching/creating user
so subsequent messages hit Step 1 directly (zero API calls)
- Pass sender_id from dingtalk_stream.py through to process_dingtalk_message
- Tenant model: add is_default field - alembic: migration to add is_default column, set earliest active tenant as default - tenants API: support set/clear is_default (platform_admin only), remove slug update logic, remove check-slug endpoint - resolve-by-domain: use is_default=True for global domain fallback instead of ORDER BY created_at - AdminCompanies: show Default badge in company list - EditCompanyModal: remove slug input, add default company toggle/indicator - i18n: add en/zh translations for new features
added 27 commits
March 31, 2026 14:45
- username 匹配逻辑支持 tenant_id=NULL 的情况 - 匹配成功后自动绑定租户到现有 web 用户 - 用户名冲突时抛 409 异常而不是自动加后缀 - 修复 zhuzhichao 重复账号问题
- 之前会把超过 8 个字符的 provider_user_id 截断（如 zhuzhichao→zhuzhich） - 现在直接使用完整的 provider_user_id 作为用户名 - 修复生产环境用户名截断问题
- 当 tenant_id 有值时，同时匹配该租户的用户和 tenant_id=NULL 的用户 - 修复早期 web 注册用户（无租户）无法通过 OAuth2 登录的问题
- 移除"此域名已启用单点登录"的蓝色提示框 - 保留 SSO 按钮和"or"分隔线 - 优化登录页面视觉效果
- 新建 backend/app/schemas/oauth2.py Pydantic 模型定义
- OAuth2FieldMapping: 4 个字段映射字段
- OAuth2Config: 7 个 OAuth2 配置字段 + URL 验证器
- OAuth2ProviderCreate/Update: 严格 config 格式模型
- 修改 backend/app/api/enterprise.py
- 导入新的 Pydantic 模型
- 添加 IdentityProviderUpdate 类
- 重写 create_oauth2_provider: 使用 OAuth2ProviderCreate
- 重写 update_oauth2_provider: 使用 OAuth2ProviderUpdate，移除兼容逻辑
- field_mapping 处理：None/null = 清空，{} = 清空，{...} = 自定义
- 修改 frontend/src/pages/EnterpriseSettings.tsx
- initOAuth2FromConfig 返回完整 OAuth2Config
- handleExpand 直接使用 config 结构
- 所有 OAuth2 字段绑定到 form.config.xxx
- 字段映射单个清空按钮 (✕)
- 字段映射全部清空按钮 (🗑️)
- Mutations 直接发送数据（无转换）
- 测试验证
- 新建/编辑 OAuth2 Provider 正常
- Scope 等字段更新正常
- 字段映射单个清空正常
- 字段映射全部清空正常
Closes: OAuth2 表单数据结构不一致问题
Restores: - HTTPException import (P0: NameError on username conflict) - OAuth2 field_mapping + _get_field system (front-end config had no effect) - get_user_info_from_token_data with _get_field support (P0: sso.py:196 call) - OAuth2 _create_new_user override (provider_user_id as username) - Feishu token Redis cache with TTL (was pure memory, no expiry refresh) - tenant_id OR NULL user matching (prevent duplicate accounts for legacy users)
Changes: - dingtalk: save mobile/email from corp API when creating new users (Step4) - dingtalk: add Step3d display_name unique match as fallback - dingtalk: enrich existing matched users with mobile/email from corp API - dingtalk: add Step3 user_detail logging for debugging - dingtalk: also check org_email field from corp API - auth_provider: add Step6 display_name unique match as fallback Covers all 6 registration order scenarios: - OAuth2 first → DingTalk (with/without mobile): merge via mobile/display_name - DingTalk first → OAuth2 (with/without mobile): merge via mobile/display_name - Single channel only: no change
Merged 88 upstream commits. Resolved 14 conflict files: - dingtalk.py: kept our cross-channel user merge implementation - sso.py: adopted upstream platform_service for base URL resolution - auth.py: merged upstream registration/email features with our tenant-scoped login check - enterprise.py: adopted upstream platform_service SSO domain auto-assign - feishu.py: adopted upstream channel_user_service (our OrgMember logic was redundant) - tenants.py: merged upstream multi-tenant self-create with our subdomain_prefix + agent seeding - agent_tools.py: adopted upstream platform_service for webhook URL - Login.tsx: adopted upstream localhost skip for tenant resolution - EnterpriseSettings.tsx: adopted upstream SsoChannelSection component, kept our i18n IDP descriptions - AgentDetail.tsx: adopted upstream chat-composer CSS refactor - zh.json: merged all i18n keys from both sides - teams.py: removed redundant agent reload (already loaded earlier) - wecom.py: adopted upstream auth_provider pattern (handles OrgMember internally) - ChannelConfig.tsx: merged imports from both sides
…platform_service - sso.py: use resolve_base_url instead of platform_service for OAuth callbacks - auth.py: use resolve_base_url for SSO redirect URL - enterprise.py: use resolve_base_url for SSO address display - platform_service.py: add system_settings DB lookup as fallback (ENV > DB > Request) - AdminCompanies.tsx: restore platform domain config UI (Public URL card)
- auth_provider: _create_new_user uses find_or_create_identity, fixed variable order bug - auth_provider: OAuth2 _create_new_user override adapted to Identity model - auth_provider: _update_existing_user operates through user.identity - auth_provider: Step5 username query via join(Identity) - auth_provider: ensure identity loaded before updating proxy fields - dingtalk: Step2/3b/3c queries via join(Identity) - dingtalk: Step4 creates Identity before User - dingtalk: update matched users through identity relationship - sso_service: match_user_by_email/mobile via join(Identity) with selectinload - sso.py: get_sso_session_status loads identity for UserOut validation - EnterpriseSettings.tsx: callback URL falls back to window.location.origin
…ations - tenants.py: strip port from domain before subdomain prefix matching - users.py: admin user edit operates through identity relationship - wecom_stream.py: user creation via find_or_create_identity - agent_tools.py: user lookup via join(Identity)
…ion_proxy lazy loading - Set User.identity relationship to lazy="selectin" for automatic eager loading - Fix chat_sessions and agent_tools using association_proxy as SQL column expression - Add missing UserModel import in dingtalk message handler - Add selectinload for creator query in agent detail endpoint
… merge resolution
Contributor
|
Thank you for this tremendous amount of work, @nap-liu! We can see you have put serious effort into this — the DingTalk multimedia support, OAuth2 SSO enhancements, multi-tenant management, and security hardening are all genuinely valuable features. After a thorough review, we are not able to merge this PR as-is for a few reasons: 1. Size and scopeAt 67 files and +4,600 lines, this is very difficult to review safely. It mixes new features, bug fixes, and architectural adaptations in a way that makes it hard to reason about the impact of each change. 2. Conflicts with current
|
wisdomqin
added a commit
that referenced
this pull request
Apr 3, 2026
…reenlet association_proxy fields (email, username, password_hash, email_verified, primary_mobile) access User.identity at attribute-read time. In async SQLAlchemy contexts this triggered a synchronous greenlet IO call, causing: sqlalchemy.exc.MissingGreenlet: greenlet_spawn has not been called Observed 63 occurrences in 10 minutes on production. Adding lazy='selectin' makes SQLAlchemy eagerly load identity alongside every User query, eliminating the lazy-load IO entirely and fixing the crash. Ref: PR #258 (nap-liu)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Comprehensive enhancements to the Clawith platform covering channel capabilities, enterprise authentication, multi-tenant management, and full adaptation for the upstream Identity architecture refactor.
📡 DingTalk Channel Enhancements
Multimedia Message Support
/newcommand to reset sessions (DingTalk and Feishu)User Matching
🔐 OAuth2 SSO Login
🏢 Multi-Tenant Management
Company Administration
Domain Resolution
resolve_base_urlUser Management
🏗 Identity Architecture Adaptation
Full adaptation after upstream introduced the global Identity system:
Core Fixes
lazy="selectin"onUser.identityrelationship to preventMissingGreenletcrashes in async contextsassociation_proxyincorrectly used as SQL column expressionsNameErrorChannel Adaptation
auth_providerlogin matching to join Identity table🔒 Security Hardening
secrets.md: creator-only sensitive file with redaction pipeline🛠 Other Improvements
feishu_doc_share→feishu_drive_share, addfeishu_drive_delete📦 Database Migrations
add_user_sourcemerge_headsadd_agent_credentials440261f5594fd9cbd43b62e5f8a934bf9f17