HIVE-27193: Database names starting with '@' cause error during ALTER…#6371
HIVE-27193: Database names starting with '@' cause error during ALTER…#6371ashniku wants to merge 1 commit intoapache:masterfrom
Conversation
|
CI is failing. Can you please fix it first ? |
|
closing the PR, I will re-open again |
|
retest this please |
|
| @@ -0,0 +1,6 @@ | |||
| CREATE DATABASE `@test`; | |||
There was a problem hiding this comment.
Please use meaningful test file names that can help understand the purpose of the qtest file in future
| @@ -0,0 +1,44 @@ | |||
| PREHOOK: query: CREATE DATABASE `@test` | |||
| CREATE DATABASE `@test`; | ||
| USE `@test`; | ||
| CREATE TABLE testtable (c1 INT); | ||
| ALTER TABLE testtable ADD COLUMNS (c2 INT); |
There was a problem hiding this comment.
Instead of using the database and alter and drop table, can you run the query as mentioned in the JIRA as I see that the issue is seen when hive tries to parse dbName.tableName where the dbName has the special character @ as it looks out for the catalog name.
Change queries to
ALTER TABLE `@test`.testtable ADD COLUMNS (c2 INT);
DROP TABLE `@test`.testtable;
You should also create another table named testtable2 and try the following queries as well
ALTER TABLE `@hive#@test`.testtable ADD COLUMNS (c2 INT);
DROP TABLE `@hive#@test`.testtable;
There was a problem hiding this comment.
@Neer393 The Jira was created
ALTER TABLE @test.testtable ADD COLUMNS (c2 INT);
DROP TABLE @test.testtable;
This above one succeeds.
I just have a doubt on below:
ALTER TABLE @hive#test.testtable ADD COLUMNS (c2 INT);
DROP TABLE @hive#test.testtable;In this case @hive#test the whole word is database or it is deciphered as catalog#database. The purpose is to check with @ as per the kita.
Could you please let me know??
There was a problem hiding this comment.
There was a problem hiding this comment.
In this case @hive#test the whole word is database or it is deciphered as catalog#database. The purpose is to check with @ as per the kita.
It is deciphered as @catalog#database
So as per this JIRA, ALTER TABLE queries are failing because they call the MetastoreUtils.parseDbName() method which is calling the MetastoreUtils.hasCatalogName() method which you have fixed but the purpose of parseDbName is that it should return an array of size 2 where the 1st element is catalog name and the 2nd element is the database name.
So just to make sure both with and without catalog name works, I have asked you to run both queries
There was a problem hiding this comment.
CREATE DATABASE `@test`;
CREATE TABLE `@test`.testtable (c1 INT);
ALTER TABLE `@hive#@test`.testtable ADD COLUMNS (c2 INT);
DROP TABLE `@hive#@test`.testtable;
DROP DATABASE `@test`;



…/DROP table
What changes were proposed in this pull request?
I have mofified "MetaStoreUtils.java", updated "hasCatalogName" to return true only if the database name both starts with the catalog marker (@) and contains the catalog-database separator (#).
-->
Why are the changes needed?
The issue is caused by MetaStoreUtils.hasCatalogName incorrectly identifying database names that start with the '@' character as catalog-prepended names. This leads to a failure in MetaStoreUtils.parseDbName when it tries to split the name using the '#' separator, which is missing in regular database names starting with '@'.
Proposed Changes
[Metastore Common]
[MODIFY] MetaStoreUtils.java
Update hasCatalogName
to return true only if the database name both starts with the catalog marker (@) and contains the catalog-database separator (#).
Does this PR introduce any user-facing change?
NO
-->
How was this patch tested?
CREATE DATABASE @test;
USE @test;
CREATE TABLE testtable (c1 INT);
ALTER TABLE testtable ADD COLUMNS (c2 INT);
DROP TABLE testtable;
DROP DATABASE @test;