[BugFix] Fix MySQL 8.0.x compatibility problem #56872

Zac-saodiseng · 2025-03-13T08:33:39Z

Why I'm doing:

Connect to the Cluster via IntelliJ (might occur on other database management tools) using a compatible 8.0.x MySQL driver
Attempt to refresh the Schema in the Database UI
Exception being thrown when trying to scan the schema using the MySQL 8.0.33 driver:

Connection Error
[HY000][1193] Getting analyzing error. Detail message:
Unknown system variable 'default_storage_engine', the
most similar variables are {'default_rowset_type',
'default_table_compression', 'enable_sort_aggregate'}.

Connection Error
Getting analyzing error. Detail message:
Column 'subpartition_ordinal_position' cannot
be resolved.
and 1 duplicate report.

Getting analyzing error. Detail message:
Unknown table 'information_schema.applicable_roles'.

Connection Error
Unknown compression type (0) backend [id=10002].

Getting analyzing error. Detail message:
Column expression cannot be resolved.

Getting analyzing error. Detail message:
Column 'c.enforced' cannot be resolved.

Getting analyzing error. Detail message:
Column 'subpartition_ordinal_position' cannot be resolved.
and 9 duplicate reports.

What I'm doing:

Need to add INFORMATION_SCHEMA.APPLICABLE_ROLES table.
Need to add the system variable 'default_storage_engine' and 'default_table_compression'.
Need to add the column 'EXPRESSION' in INFORMATION_SCHEMA.STATISTICS and 'ENFORCED' in INFORMATION_SCHEMA.TABLE_CONSTRAINTS.

mysql> select * from applicable_roles ;
+------+------+---------+--------------+-----------+-----------+--------------+------------+--------------+
| USER | HOST | GRANTEE | GRANTEE_HOST | ROLE_NAME | ROLE_HOST | IS_GRANTABLE | IS_DEFAULT | IS_MANDATORY |
+------+------+---------+--------------+-----------+-----------+--------------+------------+--------------+
| root | %    | root    | %            | root      | %         | NO           | YES        | NO           |
+------+------+---------+--------------+-----------+-----------+--------------+------------+--------------+
1 row in set (0.07 sec)

Fixes #55629

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

starrocks-cr · 2025-03-13T08:36:06Z

be/src/exec/schema_scanner/schema_applicable_roles_scanner.cpp

+    _applicable_roles_index++;
+    return Status::OK();
+}
+} // namespace starrocks


The most risky bug in this code is:
Potential out-of-bounds access due to _column_num being uninitialized, leading to unpredictable slot_id validation.

You can modify the code like this:

// Within SchemaApplicableRolesScanner constructor or initialization part: _column_num = sizeof(_s_columns) / sizeof(SchemaScanner::ColumnDesc); // Initialize _column_num // Ensure to include this initialization logic to avoid potential issues. SchemaScanner::SchemaScanner(_s_columns, _column_num);

starrocks-cr · 2025-03-13T08:36:08Z

fe/fe-core/src/main/java/com/starrocks/authorization/AuthorizationMgr.java

+        } finally {
+            roleReadUnlock();
+        }
+    }
 }


The most risky bug in this code is:
Improper handling of lock acquisition and release order might lead to deadlocks. Specifically, mixing userReadLock/Unlock with roleReadLock/Unlock without a consistent order can cause deadlock issues if another method acquires them in a conflicting order.

You can modify the code like this:

public List<RolePrivilegeCollectionV2> getApplicableRoles(UserIdentity user) throws PrivilegeException { List<RolePrivilegeCollectionV2> applicableRoles = new ArrayList<>(); // Ensure a consistent locking order by acquiring role lock before user lock roleReadLock(); try { userReadLock(); try { Set<Long> roleIds = getRoleIdsByUser(user); for (Long roleId : roleIds) { RolePrivilegeCollectionV2 rolePrivilegeCollection = roleIdToPrivilegeCollection.get(roleId); if (rolePrivilegeCollection != null) { applicableRoles.add(rolePrivilegeCollection); } } } finally { userReadUnlock(); } } finally { roleReadUnlock(); } return applicableRoles; }

This modification ensures that locks are always acquired and released in a consistent order, mitigating the risk of deadlocks.

starrocks-cr · 2025-03-13T08:36:12Z

fe/fe-core/src/main/java/com/starrocks/service/InformationSchemaDataSource.java

+        response.setRoles(roleList);
+        return response;
+    }
+
    public static TGetTemporaryTablesInfoResponse generateTemporaryTablesInfoResponse(TGetTemporaryTablesInfoRequest request)
            throws TException {
        TemporaryTableMgr temporaryTableMgr = GlobalStateMgr.getCurrentState().getTemporaryTableMgr();


The most risky bug in this code is: If getAuthDbRequestResult(request.getAuth_info()) returns a null value for authorizedDbs, calling isEmpty() on it will result in a NullPointerException.

You can modify the code like this:

public static TGetApplicableRolesResponse generateApplicableRolesResp(TGetApplicableRolesRequest request) throws TException { TGetApplicableRolesResponse response = new TGetApplicableRolesResponse(); List<TApplicableRolesInfo> roleList = new ArrayList<>(); AuthDbRequestResult result = getAuthDbRequestResult(request.getAuth_info()); if (result == null || result.authorizedDbs == null || result.authorizedDbs.isEmpty()) { response.setRoles(roleList); return response; } UserIdentity currentUser = result.currentUser; if (currentUser == null) { response.setRoles(roleList); return response; } AuthorizationMgr authMgr = GlobalStateMgr.getCurrentState().getAuthorizationMgr(); try { for (RolePrivilegeCollectionV2 role : authMgr.getApplicableRoles(currentUser)) { String roleName = role.getName(); String roleHost = "%"; String isGrantable = authMgr.isRoleGrantable(currentUser, roleName) ? "YES" : "NO"; String isDefault = authMgr.isRoleDefault(currentUser, roleName) ? "YES" : "NO"; String isMandatory = authMgr.isRoleMandatory(currentUser, roleName) ? "YES" : "NO"; TApplicableRolesInfo roleInfo = new TApplicableRolesInfo(); roleInfo.setUser(currentUser.getUser()); roleInfo.setHost(currentUser.getHost()); roleInfo.setGrantee(currentUser.getUser()); roleInfo.setGrantee_host(currentUser.getHost()); roleInfo.setRole_name(roleName); roleInfo.setRole_host(roleHost); roleInfo.setIs_grantable(isGrantable); roleInfo.setIs_default(isDefault); roleInfo.setIs_mandatory(isMandatory); roleList.add(roleInfo); } } catch (PrivilegeException e) { throw new RuntimeException(e); } response.setRoles(roleList); return response; }

imay · 2025-03-13T08:41:10Z

gensrc/thrift/Descriptors.thrift

@@ -182,6 +182,7 @@ enum TSchemaTableType {
    SCH_CLUSTER_SNAPSHOTS,
    SCH_CLUSTER_SNAPSHOT_JOBS,

+    SCH_APPLICABLE_ROLES,


New items should be added to the last.

Thanks, I've already revised it.

Signed-off-by: Zac-saodiseng <[email protected]>

github-actions · 2025-03-14T08:17:43Z

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2025-03-14T08:20:00Z

[FE Incremental Coverage Report]

✅ pass : 80 / 92 (86.96%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/authorization/RolePrivilegeCollectionV2.java	3	5	60.00%	[144, 169]
🔵	com/starrocks/service/InformationSchemaDataSource.java	28	34	82.35%	[622, 623, 628, 629, 655, 656]
🔵	com/starrocks/authorization/AuthorizationMgr.java	30	33	90.91%	[1811, 1825, 1839]
🔵	com/starrocks/catalog/system/information/ApplicableRolesSystemTable.java	12	13	92.31%	[25]
🔵	com/starrocks/qe/SessionVariable.java	2	2	100.00%	[]
🔵	com/starrocks/catalog/system/information/InfoSchemaDb.java	1	1	100.00%	[]
🔵	com/starrocks/catalog/system/information/StatisticsSystemTable.java	1	1	100.00%	[]
🔵	com/starrocks/service/FrontendServiceImpl.java	1	1	100.00%	[]
🔵	com/starrocks/catalog/system/information/TableConstraintsSystemTable.java	1	1	100.00%	[]
🔵	com/starrocks/catalog/system/information/PartitionsSystemTableSystemTable.java	1	1	100.00%	[]

github-actions · 2025-03-14T09:17:55Z

[BE Incremental Coverage Report]

✅ pass : 83 / 92 (90.22%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	be/src/exec/schema_scanner/schema_applicable_roles_scanner.cpp	78	87	89.66%	[45, 49, 54, 55, 57, 58, 86, 89, 104]
🔵	be/src/exec/schema_scanner.cpp	2	2	100.00%	[]
🔵	be/src/exec/schema_scanner/schema_helper.cpp	2	2	100.00%	[]
🔵	be/src/exec/schema_scanner/schema_applicable_roles_scanner.h	1	1	100.00%	[]

sonarqubecloud · 2025-03-14T09:58:50Z

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Astralidea · 2025-03-14T10:30:52Z

fe/fe-core/src/main/java/com/starrocks/catalog/system/SystemId.java

@@ -103,6 +103,7 @@ public class SystemId {
    public static final long FE_METRICS_ID = 42L;

    public static final long TEMP_TABLES_ID = 43L;
+    public static final long APPLICABLE_ROLES_ID = 45L;



It should be moved behind 44.

Astralidea · 2025-03-14T10:31:39Z

fe/fe-core/src/main/java/com/starrocks/authorization/RolePrivilegeCollectionV2.java

+    @SerializedName(value = "d")
+    private boolean isDefault;
+    @SerializedName(value = "m")
+    private boolean isMandatory;


It’s best not to add extra persistent fields.

Zac-saodiseng requested review from a team as code owners March 13, 2025 08:33

wanpengfei-git added META-REVIEW PROTO-REVIEW labels Mar 13, 2025

mergify bot assigned Zac-saodiseng Mar 13, 2025

wanpengfei-git requested a review from a team March 13, 2025 08:34

github-actions bot added the 3.4 label Mar 13, 2025

wanpengfei-git requested a review from a team March 13, 2025 08:34

starrocks-cr bot reviewed Mar 13, 2025

View reviewed changes

Zac-saodiseng force-pushed the fix_ide_recognition branch from fc36f03 to 4395186 Compare March 13, 2025 08:36

imay reviewed Mar 13, 2025

View reviewed changes

Zac-saodiseng added 3 commits March 13, 2025 16:49

[BugFix] Fix MySQL 8.0.x compatibility problem

3592d83

Signed-off-by: Zac-saodiseng <[email protected]>

[Test] Add fix MySQL 8.0.x compatibility problem UT

feb32a9

Signed-off-by: Zac-saodiseng <[email protected]>

[BugFix] Fix MySQL 8.0.x compatibility problem

6d06425

Signed-off-by: Zac-saodiseng <[email protected]>

Zac-saodiseng force-pushed the fix_ide_recognition branch from 88b8302 to 6d06425 Compare March 13, 2025 08:49

[BugFix] Fix MySQL 8.0.x compatibility problem

bc0b829

Signed-off-by: Zac-saodiseng <[email protected]>

Zac-saodiseng force-pushed the fix_ide_recognition branch from e95f59b to bc0b829 Compare March 14, 2025 05:45

Merge branch 'StarRocks:main' into fix_ide_recognition

334d756

Astralidea approved these changes Mar 14, 2025

View reviewed changes

Merge branch 'StarRocks:main' into fix_ide_recognition

6b89714

Astralidea reviewed Mar 14, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BugFix] Fix MySQL 8.0.x compatibility problem #56872

[BugFix] Fix MySQL 8.0.x compatibility problem #56872

Zac-saodiseng commented Mar 13, 2025 •

edited

Loading

starrocks-cr bot Mar 13, 2025

starrocks-cr bot Mar 13, 2025

starrocks-cr bot Mar 13, 2025

imay Mar 13, 2025

Zac-saodiseng Mar 13, 2025

github-actions bot commented Mar 14, 2025

github-actions bot commented Mar 14, 2025

github-actions bot commented Mar 14, 2025

sonarqubecloud bot commented Mar 14, 2025

Astralidea Mar 14, 2025

Astralidea Mar 14, 2025

[BugFix] Fix MySQL 8.0.x compatibility problem #56872

Are you sure you want to change the base?

[BugFix] Fix MySQL 8.0.x compatibility problem #56872

Conversation

Zac-saodiseng commented Mar 13, 2025 • edited Loading

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

starrocks-cr bot Mar 13, 2025

Choose a reason for hiding this comment

starrocks-cr bot Mar 13, 2025

Choose a reason for hiding this comment

starrocks-cr bot Mar 13, 2025

Choose a reason for hiding this comment

imay Mar 13, 2025

Choose a reason for hiding this comment

Zac-saodiseng Mar 13, 2025

Choose a reason for hiding this comment

github-actions bot commented Mar 14, 2025

[Java-Extensions Incremental Coverage Report]

github-actions bot commented Mar 14, 2025

[FE Incremental Coverage Report]

file detail

github-actions bot commented Mar 14, 2025

[BE Incremental Coverage Report]

file detail

sonarqubecloud bot commented Mar 14, 2025

Quality Gate passed

Astralidea Mar 14, 2025

Choose a reason for hiding this comment

Astralidea Mar 14, 2025

Choose a reason for hiding this comment

Zac-saodiseng commented Mar 13, 2025 •

edited

Loading