一、背景
一張person表,有id和name的兩個(gè)字段,id是唯一的不允許重復(fù),id相同則認(rèn)為是重復(fù)的記錄。
二、解決
select id from group by id having count(*) > 1
按照id分組并計(jì)數(shù),某個(gè)id號(hào)那一組的數(shù)量超過1條則認(rèn)為重復(fù)。
如何查詢重復(fù)的數(shù)據(jù)
1
|
select 字段1,字段2, count (*) from 表名 group by 字段1,字段2 having count (*) > 1 |
PS:將上面的>號(hào)改為=號(hào)就可以查詢出沒有重復(fù)的數(shù)據(jù)了。
Oracle刪除重復(fù)數(shù)據(jù)的SQL(刪除所有):
刪除重復(fù)數(shù)據(jù)的基本結(jié)構(gòu)寫法:
想要?jiǎng)h除這些重復(fù)的數(shù)據(jù),可以使用下面語句進(jìn)行刪除
1
2
|
delete from 表名 a where 字段1,字段2 in ( select 字段1,字段2, count (*) from 表名 group by 字段1,字段2 having count (*) > 1) |
上面的SQL注意:語句非常簡單,就是將查詢到的數(shù)據(jù)刪除掉。不過這種刪除執(zhí)行的效率非常低,對于大數(shù)據(jù)量來說,可能會(huì)將數(shù)據(jù)庫吊死。
建議先將查詢到的重復(fù)的數(shù)據(jù)插入到一個(gè)臨時(shí)表中,然后對進(jìn)行刪除,這樣,執(zhí)行刪除的時(shí)候就不用再進(jìn)行一次查詢了。如下:
CREATE TABLE 臨時(shí)表 AS (select 字段1,字段2,count(*) from 表名 group by 字段1,字段2 having count(*) > 1)
上面這句話就是建立了臨時(shí)表,并將查詢到的數(shù)據(jù)插入其中。
下面就可以進(jìn)行這樣的刪除操作了:
1
|
delete from 表名 a where 字段1,字段2 in ( select 字段1,字段2 from 臨時(shí)表); |
Oracle刪除重復(fù)數(shù)據(jù)的SQL(留下一條記錄):
oracle中,有個(gè)隱藏了自動(dòng)rowid,里面給每條記錄一個(gè)唯一的rowid,我們?nèi)绻氡A糇钚碌囊粭l記錄,我們就可以利用這個(gè)字段,保留重復(fù)數(shù)據(jù)中rowid最大的一條記錄就可以了。
使用ROWID查詢重復(fù)數(shù)據(jù):
1
2
3
4
5
|
select a.rowid,a.* from 表名 a where a.rowid != ( select max (b.rowid) from 表名 b where a.字段1 = b.字段1 and a.字段2 = b.字段2 ) |
括號(hào)中的SQL查詢出rowid最大的記錄,而外面就是查詢出除了rowid最大之外的其他重復(fù)的數(shù)據(jù)了。
由此,我們要?jiǎng)h除重復(fù)數(shù)據(jù),只保留最新的一條數(shù)據(jù),就可以這樣寫了:
刪除重復(fù)數(shù)據(jù)(留下最大ROWID的一條)
1
2
3
4
|
delete from 表名 a where a.rowid != ( select max (b.rowid) from 表名 b where a.字段1 = b.字段1 and a.字段2 = b.字段2 ) |
刪除重復(fù)數(shù)據(jù)(留下最小ROWID的一條)
1
2
3
|
delete tab t where t.rowid > ( select min (t2.rowid) from tab t2 where t.col2 = t2.col2 and t.col8 = t2.col8 ) |
當(dāng)然,上面語句的執(zhí)行效率是很低的,可以考慮建立臨時(shí)表,講需要判斷重復(fù)的字段、rowid插入臨時(shí)表中,然后刪除的時(shí)候在進(jìn)行比較。
1
2
3
4
5
6
7
|
create table 臨時(shí)表 as select a.字段1,a.字段2, MAX (a.ROWID) dataid from 正式表 a GROUP BY a.字段1,a.字段2; delete from 表名 a where a.rowid != ( select b.dataid from 臨時(shí)表 b where a.字段1 = b.字段1 and a.字段2 = b.字段2 ); commit ; |
對于完全重復(fù)記錄的刪除
對于表中兩行記錄完全一樣的情況,可以用下面語句獲取到去掉重復(fù)數(shù)據(jù)后的記錄:
1
|
select distinct * from 表名 |
可以將查詢的記錄放到臨時(shí)表中,然后再將原來的表記錄刪除,最后將臨時(shí)表的數(shù)據(jù)導(dǎo)回原來的表中。如下:
1
2
3
4
|
CREATE TABLE 臨時(shí)表 AS ( select distinct * from 表名); truncate table 正式表; insert into 正式表 ( select * from 臨時(shí)表); drop table 臨時(shí)表; |
如果想刪除一個(gè)表的重復(fù)數(shù)據(jù),可以先建一個(gè)臨時(shí)表,將去掉重復(fù)數(shù)據(jù)后的數(shù)據(jù)導(dǎo)入到臨時(shí)表,然后在從臨時(shí)表將數(shù)據(jù)導(dǎo)入正式表中,如下:
1
2
|
INSERT INTO t_table_bak select distinct * from t_table; |
MySQL查詢及刪除重復(fù)記錄的方法
查詢及刪除重復(fù)記錄的方法(例子演示)
1、查找表中多余的重復(fù)記錄,重復(fù)記錄是根據(jù)單個(gè)字段(peopleId)來判斷
1
|
select * from table where tableId in ( select tableId from table group by tableId having count (tableId) > 1) |
2、刪除表中多余的重復(fù)記錄,重復(fù)記錄是根據(jù)單個(gè)字段(tableId)來判斷,只留有rowid最小的記錄。
1
|
delete from table where tableId in ( select tableId from table group by tableId having count (tableId) > 1) and rowid not in ( select min (rowid) from table group by tableId having count (tableId )>1) |
3、查找表中多余的重復(fù)記錄(多個(gè)字段)
1
|
select * from vitae a where (a.tableId,a.seq) in ( select tableId,seq from vitae group by tableId,seq having count (*) > 1) |
4、刪除表中多余的重復(fù)記錄(多個(gè)字段),只留有rowid最小的記錄
1
2
|
delete from vitae a where (a.tableId,a.seq) in ( select tableId,seq from vitae group by tableId,seq having count (*) > 1) and rowid not in ( select min (rowid) from vitae group by tableId,seq having count (*)>1) |
5、查找表中多余的重復(fù)記錄(多個(gè)字段),不包含rowid最小的記錄
1
|
select * from vitae a where (a.tableId,a.seq) in ( select tableId,seq from vitae group by tableId,seq having count (*) > 1) and rowid not in ( select min (rowid) from vitae group by tableId,seq having count (*)>1) |
6、查詢出一個(gè)表中,某一列的值是相同的SQL:
比方說
在A表中存在一個(gè)字段“name”,而且不同記錄之間的“name”值有可能會(huì)相同,
現(xiàn)在就是需要查詢出在該表中的各記錄之間,“name”值存在重復(fù)的項(xiàng);
1
|
Select Name , Count (*) From A Group By Name Having Count (*) > 1 |
如果還查性別也相同大則如下:
1
|
Select Name ,sex, Count (*) From A Group By Name ,sex Having Count (*) > 1 |
oracle查詢表中字段里數(shù)據(jù)是否有重復(fù)
查單個(gè)字段:
1
|
SELECT TEST_NAME, COUNT (*) FROM T_TEST GROUP BY TEST_NAME HAVING COUNT (*) > 1 |
Oracle查詢重復(fù)數(shù)據(jù)并刪除,只保留一條記錄
1、查找表中多余的重復(fù)記錄,重復(fù)記錄是根據(jù)單個(gè)字段(Id)來判斷
1
2
3
|
select * from 表 where Id in ( select Id from 表 group byId having count (Id) > 1) |
2、刪除表中多余的重復(fù)記錄,重復(fù)記錄是根據(jù)單個(gè)字段(Id)來判斷,只留有rowid最小的記錄
1
2
3
4
|
DELETE from 表 WHERE (id) IN ( SELECT id FROM 表 GROUP BY id HAVING COUNT (id) > 1) AND ROWID NOT IN ( SELECT MIN (ROWID) FROM 表 GROUP BY id HAVING COUNT (*) > 1); |
3、查找表中多余的重復(fù)記錄(多個(gè)字段)
1
2
3
|
select * from 表 a where (a.Id,a.seq) in ( select Id,seq from 表 group by Id,seq having count (*) > 1) |
4、刪除表中多余的重復(fù)記錄(多個(gè)字段),只留有rowid最小的記錄
1
2
3
4
|
delete from 表 a where (a.Id,a.seq) in ( select Id,seq from 表 group by Id,seq having count (*) > 1) and rowid not in ( select min (rowid) from 表 group by Id,seq having count (*)>1) |
5、查找表中多余的重復(fù)記錄(多個(gè)字段),不包含rowid最小的記錄
1
2
3
4
|
select * from 表 a where (a.Id,a.seq) in ( select Id,seq from 表 group by Id,seq having count (*) > 1) and rowid not in ( select min (rowid) from 表 group by Id,seq having count (*)>1) |
比如現(xiàn)在有一人員表 (表名:peosons)
若想將姓名、身份證號(hào)、住址這三個(gè)字段完全相同的記錄查詢出來
1
2
3
4
5
|
select p1.* from persons p1,persons p2 where p1.id<>p2.id and p1.cardid = p2.cardid and p1.pname = p2.pname and p1.address = p2.address |
可以實(shí)現(xiàn)上述效果。
幾個(gè)刪除重復(fù)記錄的SQL語句
1.用rowid方法
2.用group by方法
3.用distinct方法
1。用rowid方法
據(jù)據(jù)oracle帶的rowid屬性,進(jìn)行判斷,是否存在重復(fù),語句如下:
查數(shù)據(jù):
1
2
|
select * from table1 a where rowid !=( select max (rowid) from table1 b where a.name1=b.name1 and a.name2=b.name2……) |
刪數(shù)據(jù):
1
2
|
delete from table1 a where rowid !=( select max (rowid) from table1 b where a.name1=b.name1 and a.name2=b.name2……) |
2.group by方法
查數(shù)據(jù):
select count(num), max(name) from student
--列出重復(fù)的記錄數(shù),并列出他的name屬性
group by num
having count(num) >1 --按num分組后找出表中num列重復(fù),即出現(xiàn)次數(shù)大于一次
刪數(shù)據(jù):
1
2
3
|
delete from student group by num having count (num) >1 |
這樣的話就把所有重復(fù)的都刪除了。
3.用distinct方法 -對于小的表比較有用
1
2
3
4
|
create table table_new as select distinct * from table1 minux truncate table table1; insert into table1 select * from table_new; |
查詢及刪除重復(fù)記錄的方法大全
1、查找表中多余的重復(fù)記錄,重復(fù)記錄是根據(jù)單個(gè)字段(peopleId)來判斷
1
2
3
|
select * from people where peopleId in ( select peopleId from people group by peopleId having count (peopleId) > 1) |
2、刪除表中多余的重復(fù)記錄,重復(fù)記錄是根據(jù)單個(gè)字段(peopleId)來判斷,只留有rowid最小的記錄
1
2
3
4
5
6
|
delete from people where peopleId in ( select peopleId from people group by peopleId having count (peopleId) > 1) and rowid not in ( select min (rowid) from people group by peopleId having count (peopleId )>1) |
3、查找表中多余的重復(fù)記錄(多個(gè)字段)
1
2
3
|
select * from vitae a where (a.peopleId,a.seq) in ( select peopleId,seq from vitae group by peopleId,seq having count (*) > 1) |
4、刪除表中多余的重復(fù)記錄(多個(gè)字段),只留有rowid最小的記錄
1
2
3
4
5
|
delete from vitae a where (a.peopleId,a.seq) in ( select peopleId,seq from vitae group by peopleId,seq having count (*) > 1) and rowid not in ( select min (rowid) from vitae group by peopleId,seq having count (*)>1) |
5、查找表中多余的重復(fù)記錄(多個(gè)字段),不包含rowid最小的記錄
1
2
3
4
5
|
select * from vitae a where (a.peopleId,a.seq) in ( select peopleId,seq from vitae group by peopleId,seq having count (*) > 1) and rowid not in ( select min (rowid) from vitae group by peopleId,seq having count (*)>1) |
(二)
比方說
在A表中存在一個(gè)字段“name”,
而且不同記錄之間的“name”值有可能會(huì)相同,
現(xiàn)在就是需要查詢出在該表中的各記錄之間,“name”值存在重復(fù)的項(xiàng);
1
2
|
Select Name , Count (*) From A Group By Name Having Count (*) > 1 |
如果還查性別也相同大則如下:
1
2
|
Select Name ,***, Count (*) From A Group By Name ,*** Having Count (*) > 1 |
(三)
方法一
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
declare @ max integer ,@id integer declare cur_rows cursor local for select 主字段, count (*) from 表名 group by 主字段 having count (*) >; 1 open cur_rows fetch cur_rows into @id,@ max while @@fetch_status=0 begin select @ max = @ max -1 set rowcount @ max delete from 表名 where 主字段 = @id fetch cur_rows into @id,@ max end close cur_rows set rowcount 0 |
方法二
"重復(fù)記錄"有兩個(gè)意義上的重復(fù)記錄,一是完全重復(fù)的記錄,也即所有字段均重復(fù)的記錄,二是部分關(guān)鍵字段重復(fù)的記錄,
比如Name字段重復(fù),而其他字段不一定重復(fù)或都重復(fù)可以忽略。
1、對于第一種重復(fù),比較容易解決,使用
select distinct * from tableName就可以得到無重復(fù)記錄的結(jié)果集。
如果該表需要?jiǎng)h除重復(fù)的記錄(重復(fù)記錄保留1條),可以按以下方法刪除
1
2
3
4
|
select distinct * into #Tmp from tableName drop table tableName select * into tableName from #Tmp drop table #Tmp |
發(fā)生這種重復(fù)的原因是表設(shè)計(jì)不周產(chǎn)生的,增加唯一索引列即可解決。
2、這類重復(fù)問題通常要求保留重復(fù)記錄中的第一條記錄,操作方法如下
假設(shè)有重復(fù)的字段為Name,Address,要求得到這兩個(gè)字段唯一的結(jié)果集
1
2
3
4
5
6
|
select identity( int ,1,1) as autoID, * into #Tmp from tableName select min (autoID) as autoID into #Tmp2 from #Tmp group by Name ,autoID select * from #Tmp where autoID in ( select autoID from #tmp2) |
最后一個(gè)select即得到了Name,Address不重復(fù)的結(jié)果集(但多了一個(gè)autoID字段,實(shí)際寫時(shí)可以寫在select子句中省去此列)
(四)查詢重復(fù)
1
2
3
4
5
|
select * from tablename where id in ( select id from tablename group by id having count (id) > 1 ) |
原文鏈接:http://www.cnblogs.com/qmfsun/p/4871776.html