Karaoke Forums

Well, I've been buying, and ripping, and making "new" books (again). SOOOO, I decided that I have enough songs that I can start printing books without DUPLICATES. I have in "my" opinion 2 of the best programs to de-duplicate, I use Fasttracks V2 songbook creator for my books, and now the NEW version of Compuhost will show only one of each DUPLICATE title unless you choose to see all of them. WELL, just goes to show you how NIEVE I am. OMG!!!! AFTER letting the programs DE-DUPLICATE, I decided to check, and correct ANY DUPLICATES that WEREN'T caught. THREE DAYS later, I FINALLY finished de-duplicating songs,(by artist). EACH manufacturer has their own way of listing the names, AND the name of the songs. I have about 14,500 songs (including dupes), 8,450 WITHOUT DUPES. I had NO idea that it would be such a MAJOR undertaking, but I'm glad I've done it. I am a hunt and peck typist, so I can ONLY IMAGINE how long it would have taken if the PROGRAMS hadn't de-duplicated the EXACT DUPLICATES. Has anyone else tried this, and if so, was it as hard for you, just curious???

Rosario

Serenity Now Karaoke

I saw a similar reduction in numbers from nearly 14,000 to 9000

I did my songbooks in excel so I went through pretty much manually to make sure all of the times an artist and song was written it is printed the same way.

Unfortunately by de-duping you cant really list the manus of the songs. I keep all of my songs in compuhost, so I can choose the manu on the fly. I usually go with SC, Zoom, SBI, Sunfly, Dangerous, PHM, SGB, MM in about that order. But rarely if a song is a bad version It is good to have a backup of another manu.

I figure it pays for itself though in the cost of songbooks. Songbooks cost me about $20 each and would be $10 more with the dups due to extra pages. Since I redo 6 songbooks every 3 months or so that adds up to around 240 a year. Certainly worth a few hours of editing time at home.

I went from approx 14,500 dup'd to close to 13,000 non-dup'd. All 'de'dup'd' manually through Access, don't care for songbook makers in general as they are usually wrong in SOME repect - as you quickly found.
Still have everything in the computer so if anyone has a preference they can ask.

_________________
LIKE Lonman on Facebook - Lonman Productions Karaoke & my main site via my profile!

I am currently undertaking this task and have been on and off it for the last 3 months. I really need to just get to it, but I seem to have to take breaks fairly often.
I am using a pdf copy of my book listed by artist and KJ File Manager.I am going through each one manually and culling out typos as well, which I've not done to any extent. It's turned out to be a much bigger job than I first thought.
Maybe I'll pick it up again tomorrow....

I use excel and went through song by song selecting the best versions (I think). The reason for this is there may be a slow and fast version of a song or radio cuts, etc which you may want to keep in. A program may take out other artist versions. It also gives you a chance to make sure titles (and artists) are correct and spelled right. It may take more time but you'll probably be happier with the results.

_________________
You can be strange but not a stranger

'de'dup'd'

That needs to be in some kind of karaoke dictionary!

I don't care for songbook makers either, and do everything manually through Access as well for exactly the same reason. It's been labor-intensive but I always feel it's worth it when I print and books and browse them for review and accuracy. The first time I 'de'dup'd' I really didn't have many songs total, and it was still labor intensive. My issue then was I could see myself running out of hard-drive space soon so the duplicates were unnecessary files, and since then I still usually only keep a file for one version of each song to play from. It felt like I cut my already small book in half! I immediately felt the saving in less paper, though, and it evaded a coming costly change in binder size for a while.

The first go-round I just deleted the tracks from the database if I wasn't keeping them. Then I realized if I needed to use the other version off the disc for some reason I could no longer look it up in Access. DOH!

After that I added another column in Access to track the status of the track: the one I use, one to definitely not use (where I've compared it to the version in use), one that's likely an exact duplicate of the track in use, one I may use, etc. I've come up with a few random changes in the codes I use there over the years, and update that field as I add more songs and convert more discs.

Right now I'm adding some more discs into Access (I found The Singing Machine #G9921 Nickeloden for $1.50 used at Goodwill 50% off day! SPONGEBOB SQUAREPANTS! ) and I'm at about 19,000 tracks, but that includes LDs & DVDs and other odd formats I've cataloged but may never find the time to convert to files, and some unconverted CDGs. My last Artist book showed 10,245 tracks including duplicates I deemed worth keeping and cross-listings.

The thought of trying to do it all at once from scratch now manually totally scares the hell out of me...

For sure; the difference it makes is well worth the work. That's exactly why there are a few duplicates I keep. Like for example: Ike & Tina Turner Proud Mary. DK is noted as "(with spoken intro)" and Priddis is noted as "(without spoken intro)". There's no automated program that would do that.

I'm in the process of doing that too with a big exception. I'm getting rid of every song newer than 1989.

_________________
The Line Array Experiment is over. Nothing to see here. Move along.

I don't want to derail this thread, but why would you do that?

About the OP, the only way to effectively de-dupe is manually. I do it in Excel after sorting by title and then by artist. Sometimes song titles and artist names aren't spelled exactly the same. You can have "Joy To The World" by 3 Dog Nite and Three Dog Night. An automated script probably won't have that kind of built-in logic.

_________________
[font=Lucida Console]DangerousKaraoke.com[/font]
[font=Lucida Console]"Sing for the day, sing for the moment, sing for the time of your life!"[/font]

It's for a specific bar. The bar owner wants only 60's 70's and 80's songs. It's the only way to guarantee that singers will chose from there. I'm not getting rid of genres just age of songs. I'll use my songlist station at my other gigs.

_________________
The Line Array Experiment is over. Nothing to see here. Move along.

OK my question is by Dedupe do you you mean getting rid of songs by the same artist with the same manufacture or any manufacture?

Or same song titles by different artists.

Or just what?

If you are deleting songs by the same artist with the same manufacture then I use a program called Clone Spy. It finds things on the hard drive that are the same title (ex. Garth Brooks - Friends In Low Places - CB) and you can delete which ever one you want.

Of course this only works for digital KJ's and the titles have to be named the same after doing this I dropped 1000 songs that I had dupes to.

If you are deleting songs by the same artist different manu's the question is why?
Or songs titles by different artists same question.

_________________
I like everyone when I first meet them. If you don't like me that's not my problem it's YOURS!
A stranger is a friend you haven't met yet

I am deleting duplicates of the same song same manu same artist. The problem using a program is that I have spelling mistakes and even in some cases wrong artists, title mistakes, etc. It's what happens when you manually name each song after ripping like I did a few years back before Power CDG Ripper. Even with that program, however, mistakes are made.

Correct - same song titles from the same artists from all manus.

No a cover tune by a different artist of the same song may be a completely different rendition. Example
Sweet Dreams Are Made Of These - Eurythmics
Sweet Dreams Are Made Of These - Marilyn Manson
Same song lyric wise and even a hint of melody, but completely different renditions.


The reason why is to only list 1 title of each song I actually have - not list 10 versions of Friends In Low Places. I will list the the album version & the live version (with added verse) because they are different enough & some people do not like the longer version. Or I have At This Moment with a slow beat count ending vs one that just goes through fast at the end - both of these get listed.
For me it's not a huge issue, my duplicate ratio is pretty low to begin with. I originally did it mainly for when I listed every version, 9 out of 10 times someone would ask me what the difference between them were in which I explained the manufacturers are like a cover band, some are better than others - in which their answer was ALWAYS - just give me whatever is the best - in essenece leaving it up to me anyway, so I went ahead and made my books reflect one version only on what I felt was the better version. I still keep a master book with everything listed with me, and if someone asks if I have a particular version, they are still welcome to use that one.

_________________
LIKE Lonman on Facebook - Lonman Productions Karaoke & my main site via my profile!

This is exactly what I am aiming to achieve and for the same reason. I can count on one hand how many times I've been asked for a specific version.
Most simply don't care. Those that do will know to ask.
This will save me tons in paper and toner, and it will be easier to bind. I am going with the Staples spiral binding service with a laminated front and back cover.
Much cheaper than the regular binders and sheet protectors.

Facts that would probably be helpful before anyone get's to the "de-duping" (love that term) step

1. All information, whether on the crystal case, disc or coming from the internet, must be reviewed before being added to the catalog in the first place. This will minimize dups!
a) spelling
b) correct name (3 Dog vs Three Dog)
c) it follows convention (The Beatles, vs Beatles, The)
d) use of the vernacular (I'm Not Gonna Do Anything Without You vs I Am Not Going to Do Anything Without You, vs I'm Not Gonna Do Anythin' Without You, vs I'm Not Going to Do Anythin' Without You, etc)

NOTE: I always leave out the "g" or the " ' " for words that end in "ing", particularly words like "anything", "something", "nothing"!

Also, be careful of fonts. I have copied file names off of the internet and pasted them onto the zip file. When reading the files, my programs have looked at two files with the same name (spelling and punctuation) to be different because the font of the punctuation was different from the font used on another file!

To answer Lone Wolf's question, I'm not "deleting" any versions of any songs. I am keeping ALL versions of my songs. I'm de-duplicating song TITLES, so that each title appears only once in my song BOOKS. Then in my Compuhost songlist, there is a feature where the song TITLES that are IDENTICAL can be hidden, no matter what manu-version it is, as long as the TITLE, and ARTIST are EXACTLY the same. That way, when I search as I type it will bring up the title, and I can EXPAND the song title, to reveal ALL versions of that song. The thing is, is that if ANYTHING is different, it becomes a DIFFERENT song. I guess I'm just the type of person that likes to make ALOT of EXTRA work for myself, hoping that it will help me later on, WHEN I'M DONE!!!

Rosario

If I'm in doubt of a song title or spelling then I will go to the artist's web page or see how it is spelled on the album/CD. Again time consuming at times but worth it.

_________________
You can be strange but not a stranger

Or Wikkipedia is a good source of info.

_________________
The Line Array Experiment is over. Nothing to see here. Move along.

I delete different same song same artist by different manufacturer tracks to save space (on my hard-drive and paper in my book) and also so the same version (optimistically the best one I own) is always consistently used for that song. I will keep different versions is there is a significant obvious reason to do so, such as I mentioned earlier with Ike & Tina Turner "Proud Mary".

As for deleting same song titles by different artists, I don't do that, but I do try and research those to ensure accuracy and make sure it's not the result of a karaoke disc manufacturer mis-crediting the original artist the track emulates. An example I found recently that comes to mind is "I'm A King Bee / Back Door Man (medley)", which I have on 3 CDGs. The instrumental and lyrics were identical on each disc (SGB, All Hits 2000, Karaoke Hits). One of the discs had it credited as Slim Harpo. A little research found this: While Slim Harpo wrote the song, his recording and arrangement sounded NOTHING like the track on the CDG that claimed it was his version. The Other CDSs credited it to Frank Marino, which was kind of right: It's a perfect karaoke track for Mahogany Rush which was fronted by Frank Marino and on some official releases is credited as "Frank Marino & Mahogany Rush". I adjusted my database to list it as "Mahogany Rush (Frank Marino)" and "Marino, Frank & Mahogany Rush" as an alternate listing, and deleted all references to Slim Harpo.

Yes, I am a little obsessive about my karaoke

Same here, sort of. I was a karaoke junkie for almost ten years before I fell into hosting.
(how that happened is a long weird story, but I replaced this guy and took over the system and library he was unable to take with him: http://www.crimezzz.net/serialkiller_ne ... S_cory.php Suffice it to say I'm nicer to my singers than he was!)

I was once the newbie who walked up to the host asking the difference, and also simply wanting the best. Unfortunately much more often than not the host had no idea! It was all disc and tracks numbers back them and after a while I figured out that, at most places I went, disc numbers 01-99 were DK and I sounded good with those versions. (I used to often start my karaoke night by writing Dan Dan - 40-08 on a slip, turning it in, and grabbing a book)

When I fell into the KJ gig, I wanted to eliminate that issue, and have.

The few people who have asked for specific versions get one of three answers:
1) "I use (x) and not (SC) because that's the disc I own. If you want to buy me the original SC disc I'll happily change it for you." This is usually followed by "Can't you just download it", and I reply "probably, but I won't."
2) "My version is better, try it and let me know what you think" (I only say that if I'm familiar with both versions and really feel that way)
3) "I think I have that version at home but not here. Thanks for the heads-up; I'll check it and let you know when you come back next show"
Answers 2 & 3 make people happy, especially 3 if I find the singer was right and switch out my active version before I see them again. Answer 1 annoys people, but we all know it's the right answer and aren't gonna let this thread go there.



This is a good off topic....

It does save bigtime on paper and toner. Also on binders and sheet-protectors, as well as time in printing and assembling the books.

As to if Staples spiral binding w/ lamination is cheaper than binders and sheet protectors over the long run is debatable. Binders and sheet protectors are recyclable, spiral binding is not. I have sheet protectors I acquired use that I have personally used in my books well over 3 years. When a binder wears out, the pages are easily moved to another binder. Additional sections can be easily added or removed. (I keep my green holiday song pages in only from Thanksgiving to New Years. I've added sections for Duets & Spanish and am considering one for Showtunes) I update frequently, and recycle and reuse everything that's in suitable condition to do so. When a drunk a$$h0le rips a page out of my book or writes on it, replacement of that page is easy. With spiral binding, the entire costly book is ruined. Some food for thought.

That reminds me, I have 3 books here at home for repair of some kind I keep ignoring...