Censoring a document

[Galik]

One thing to remember is that you are reading each bad word, into badWords using getline(). This means that your words.txt file should only contain one bad word per line.

The code that I posted works on this data:

good1 good2 good3
bad4 bad5 good6 bad7
good8 bad9

Using this words.txt

bad1
bad2
bad3
bad4
bad5
bad6
bad7
bad8
bad9

To produce this:

good1 good2 good3
**** **** good6 ****
good8 ****

[Max Edgar]

My file to be censored consists of this

So she set the little creature down, and felt quite relieved to see it trot away quietly into the
wood. “If it had grown up,” she said to herself, “it would have made a dreadfully ugly child: but it
makes rather a handsome pig, I think.” And she began thinking over other children she knew, who might
do very well as pigs, and was just saying to herself, “if one only knew the right way to change
them--” when she was a little startled by seeing the Cheshire Cat sitting on a bough of a tree a few
yards off.

The Cat only grinned when it saw Alice. It looked good-natured, she thought: still it had very long
claws and a great many teeth, so she felt that it ought to be treated with respect.

“Cheshire Puss,” she began, rather timidly, as she did not at all know whether it would like the
name: however, it only grinned a little wider. “Come, it’s pleased so far,” thought Alice, and she
went on. “Would you tell me, please, which way I ought to go from here?”

“That depends a good deal on where you want to get to,” said the Cat.

“I don’t much care where--” said Alice.

“Then it doesn’t matter which way you go,” said the Cat.

“--so long as I get somewhere,” Alice added as an explanation.

“Oh, you’re sure to do that,” said the Cat, “if you only walk long enough.”

Alice felt that this could not be denied, so she tried another question. “What sort of people live
about here?”

“In that direction,” the Cat said, waving its right paw round, “lives a Hatter: and in that
direction,” waving the other paw, “lives a March Hare. Visit either you like: they’re both mad.”

“But I don’t want to go among mad people,” Alice remarked.

“Oh, you can’t help that,” said the Cat: “we’re all mad here. I’m mad. You’re mad.”

“How do you know I’m mad?” said Alice.

“You must be,” said the Cat, “or you wouldn’t have come here.”

Alice didn’t think that proved it at all; however, she went on “And how do you know that you’re mad?”

“To begin with,” said the Cat, “a dog’s not mad. You grant that?”

“I suppose so,” said Alice.

“Well, then,” the Cat went on, “you see, a dog growls when it’s angry, and wags its tail when it’s
pleased. Now I growl when I’m pleased, and wag my tail when I’m angry. Therefore I’m mad.”

“I call it purring, not growling,” said Alice.

“Call it what you like,” said the Cat. “Do you play croquet with the Queen to-day?”

“I should like it very much,” said Alice, “but I haven’t been invited yet.”

“You’ll see me there,” said the Cat, and vanished.

Alice was not much surprised at this, she was getting so used to queer things happening. While she
was looking at the place where it had been, it suddenly appeared again.

“By-the-bye, what became of the baby?” said the Cat. “I’d nearly forgotten to ask.”

“It turned into a pig,” Alice quietly said, just as if it had come back in a natural way.

“I thought it would,” said the Cat, and vanished again.

Alice waited a little, half expecting to see it again, but it did not appear, and after a minute
or two she walked on in the direction in which the March Hare was said to live. “I’ve seen hatters
before,” she said to herself; “the March Hare will be much the most interesting, and perhaps as
this is May it won’t be raving mad--at least not so mad as it was in March.” As she said this,
she looked up, and there was the Cat again, sitting on a branch of a tree.

“Did you say pig, or fig?” said the Cat.

“I said pig,” replied Alice; “and I wish you wouldn’t keep appearing and vanishing so suddenly: you
make one quite giddy.”

“All right,” said the Cat; and this time it vanished quite slowly, beginning with the end of the
tail, and ending with the grin, which remained some time after the rest of it had gone.

“Well! I’ve often seen a cat without a grin,” thought Alice; “but a grin without a cat! It’s the
most curious thing I ever saw in my life!”

She had not gone much farther before she came in sight of the house of the March Hare: she thought
it must be the right house, because the chimneys were shaped like ears and the roof was thatched
with fur. It was so large a house, that she did not like to go nearer till she had nibbled some
more of the lefthand bit of mushroom, and raised herself to about two feet high: even then she
walked up towards it rather timidly, saying to herself “Suppose it should be raving mad after all!
I almost wish I’d gone to see the Hatter instead!”




My badwords are

Mushroom

Appeared

Rabbit

March

Fig

Turtle

Tail

Lefthand

Pig

Tea

Pigs

Cat

Hookah

Claws

Purring




and what i get for output is


*********************************************
************************************************************************************************************************

****************************************************************************************************************

******************************************************************************

************************************************************

*********************************

************************************************************

[Max Edgar]

String.replace says
replaces the templength charactors in words begining at position x

does this mean it replaces the word begining at position x for the length of the charactors(templength), i thought the word was 1 position. meaning that it would replace each word after that position templength times?

[Max Edgar]

Ok i figured it out, i was using the wrong document, but another trouble ive found

the bad words are still the same and so is the words to be censored but here is the output, it works correctly its just inserting ********* at the beggining of sentances.



So she set the little creature down, and felt quite relieved to see it trot away quietly into the
wood. “If it had grown up,” "she" said to herself, “it would have made a dreadfully ugly child: but it
makes rather a handsome pig, I think.” And she began thinking over other children she knew, who might
do very well as pigs, and was just saying to herself, “if one only knew the right way to change
*********them--” when she was a little startled by seeing the Cheshire *** sitting on a bough of a tree a few
yards off.

*********The *** only grinned when it saw Alice. It looked good-natured, she thought: still it had very long
claws and a great many teeth, so she felt that it ought to be treated with respect.

“Cheshire Puss,” she began, rather timidly, as she did not at all know whether it would like the
name: however, it only grinned a little wider. “Come, it’s pleased so far,” thought Alice, and she
went on. “Would you tell me, please, which way I ought to go from here?”

*********“That depends a good deal on where you want to get to,” said the ***.

“I don’t much care where--” said Alice.

*********“Then it doesn’t matter which way you go,” said the ***.

“--so long as I get somewhere,” Alice added as an explanation.

*********“Oh, you’re sure to do that,” said the ***, “if you only walk long enough.”

Alice felt that this could not be denied, so she tried another question. “What sort of people live
about here?”

*********“In that direction,” the *** said, waving its right paw round, “lives a Hatter: and in that
*******************************************************direction,” waving the other paw, “lives a ***** Hare. Visit either you like: they’re both mad.”

“But I don’t want to go among mad people,” Alice remarked.

*********“Oh, you can’t help that,” said the ***: “we’re all mad here. I’m mad. You’re mad.”

“How do you know I’m mad?” said Alice.

*********“You must be,” said the ***, “or you wouldn’t have come here.”

Alice didn’t think that proved it at all; however, she went on “And how do you know that you’re mad?”

*********“To begin with,” said the ***, “a dog’s not mad. You grant that?”

“I suppose so,” said Alice.

*********“Well, then,” the *** went on, “you see, a dog growls when it’s angry, and wags its tail when it’s
pleased. Now I growl when I’m pleased, and wag my tail when I’m angry. Therefore I’m mad.”

“I call it purring, not growling,” said Alice.

*********“Call it what you like,” said the ***. “Do you play croquet with the Queen to-day?”

“I should like it very much,” said Alice, “but I haven’t been invited yet.”

*********“You’ll see me there,” said the ***, and vanished.

Alice was not much surprised at this, she was getting so used to queer things happening. While she
was looking at the place where it had been, it suddenly appeared again.

*********“By-the-bye, what became of the baby?” said the ***. “I’d nearly forgotten to ask.”

“It turned into a pig,” Alice quietly said, just as if it had come back in a natural way.

*********“I thought it would,” said the ***, and vanished again.

Alice waited a little, half expecting to see it again, but it did not appear, and after a minute
*******************************************************or two she walked on in the direction in which the ***** Hare was said to live. “I’ve seen hatters
*******************************************************before,” she said to herself; “the ***** Hare will be much the most interesting, and perhaps as
*******************************************************this is May it won’t be raving mad--at least not so mad as it was in *****.” As she said this,
*********she looked up, and there was the *** again, sitting on a branch of a tree.

*********“Did you say pig, or fig?” said the ***.

“I said pig,” replied Alice; “and I wish you wouldn’t keep appearing and vanishing so suddenly: you
make one quite giddy.”

*********“All right,” said the ***; and this time it vanished quite slowly, beginning with the end of the
tail, and ending with the grin, which remained some time after the rest of it had gone.

“Well! I’ve often seen a cat without a grin,” thought Alice; “but a grin without a cat! It’s the
most curious thing I ever saw in my life!”

*******************************************************She had not gone much farther before she came in sight of the house of the ***** Hare: she thought
it must be the right house, because the chimneys were shaped like ears and the roof was thatched
with fur. It was so large a house, that she did not like to go nearer till she had nibbled some

[Galik]

Your words.txt file has lots of empty lines in it. I think the empty lines are being read in as bad words. If you remove all the empty lines, your output should improve:

words.txt

Mushroom
Appeared
Rabbit
March
Fig
Turtle
Tail
Lefthand
Pig
Tea
Pigs
Cat
Hookah
Claws
Purring

EDIT:

Alternatively you could ignore empty lines while reading in the bad words file.

[Azagaros]

if you had read the code I supplied, it would handle the entire structure...

a snip from the code before...

for find a word in the line.....

                                      // this will find all occurrences of said word in one line...
                                      // pCurPos is the current line position of the pattern match of the word.  
                                      // replacement is the Replacement piece
                      
                                      pCurPos = CurrentLine.find(badwordlist[pListPos]);
                                      while(pCurPos != std::string::npos)
                                      {
                                            // set up the replacement piece
                                            replace.assign("*", badwordlist[pListPos].length() );
                                            // replace in the line
                                            // following line will replace one occurrence at the current position
                                            // std::string.replace took three parameters in this case.
                                            // first one was where in string I wanted to replace said string..
                                            // second was how long was it.
                                            // third was the Actual replace string.
                                            Currentline.replace(pCurPos, replace.length(), replace);
                                            // find the next position of the current bad word starting at the last known point
                                            pCurPos = CurrentLine.find(badwordlist[pListPos], pCurPos); 
                                      } // while pCurPos 

I hadn't realized I gave you the entire algorithm you were trying to accomplish just learn the tools you have in the classes and you be fine.

btw: my thoughts wouldn't care if I had a blank line or not.

[Max Edgar]

O i had read it Azagaros, and greatly appreaciate your help. The syntax in your code are still out of my scope as of this moment, i beleived i could still accomplish my goal building up the program in the manner i was, without destroying my brain in the process. I have learned so much from those who've helped me here, much more than class itself.

What do you mean Azagaros, you didnt mean to write the entire program i needed? If thats the case(no pun intended) I wouldnt think of stealing the whole thing. I am very interested in the way you accomplished the task, but implementing parts of the code into my code would result in a revamp of my whole program. I have been looking at the syntax of some of the code you have but havent yet been able to interperet it fully.(mostly the while parameter :D)

yes galik i will see what I can do about the spaces perhaps somehow ignore them.
It seems that its only doing it in an increment of 9, a clue to the problem I will keep looking

and thanks again!!!!

[Max Edgar]

Another thing i didnt factor in case sensitivity, I assume now that when i search the string it is case sensitive, i cant think how i could induce case sensitivity in the mannar i wrote the program due to the fact that im searching for the entire word not letters

[Azagaros]

I didn't write the entire code in the post but enough to do it.
ApproachOne function in the original post is the entire algorithm you are trying to accomplish here. I wrote it off the cuff and it hadn't been tested.

Building the first list in memory from the file of words given, was in the other function of BuildBadWords in the post. I try to write self documenting code and if you read it as such it might help. It also helps to learn that too.

How do you figure this is above your level? The code pieces would solve your problem.

[Galik]

One way to make your program case insensitive would be to convert all your bad words to lowercase when you read them in and make an all lowercase copy of each line you want censored. Then you compare the lowercase bad word against the lowercase copy of the line to find the position of the replacements, but remember to actually do the replacement in the original line rather than the lowercase copy and write the original line out.

I hope that makes sense.

[andywestken]

I think this an interesting problem to look at, but it is also quite involved. Is this an assignment, or an exercise from a book, or just something interesting to look at?

The solution as it stands is going to replace bad words which are substrings of innocent ones, so words like scrap and farther could be mis-censored. I think it would be better to split into words and then censor. But even this might not be enough, if you want to trap made-up words assembled out of smaller ones.

This kind of problem is a case in which I would not use std::getline, as text files often use newlines only to mark the end of paragraphs, relying on editors' word wrap capabilities to display the text correctly. Now you could use istream::get with a std::vector<char> and then process the chars. But when I am processing a stream's chars, that's when I turn to istream_iterator.

You iterate through the chars in the stream: if the character is part of a word, append it to a string; if not, process and display the current word and then reset the string. There is one last call to the process and display to account for a word right at the end of a file.

Andy

P.S. For example, where:
- the Censor class has a member variable storing the list of bad words
- appendToWord adds the char to a string member
- flushWord writes either the word or a string of *s to the o/p stream
- isPartOfWord decides whether char is part of word or not (are hyphens part of a word?)

void Censor::purifyStream(std::istream& is, std::ostream& os)
{
    // turn off skip white space
    is >> std::noskipws;

    std::istream_iterator<char> iter(is);
    const std::istream_iterator<char> iterEnd;

    while(iterEnd != iter)
    {
        char ch = *iter;

        if(isPartOfWord(ch))
        {
            appendToWord(ch);
        }
        else
        {
            flushWord(os);

            os << ch;
        }

        ++iter;
    }

    flushWord(os);
}